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A  study  was  conducted  to  assess  the  effects  of  correlated  abilities  on 
test  characteristics,  and  to  explore  the  effects  of  correlated  abilities  on 
the  use  of  a  multidimensional  item  response  theory  model  which  fioes  not 
explicitly  account  for  such  a  correlation.  Two  tests  were  constructed. 

One  test  had  two  relatively  unidimensional  subsets  of  items,  while  the 
other  had  items  that  were  all  two-dimensional.  For  each  te‘,t  response  data 
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were  generated  according  to  a  multidimensional  two-parameter  logistic 
model  using  four  groups  of  examinees.  The  groups  of  examinees  differed 
in  the  degree  of  inter-dimension  ability  correlation. 

To  evaluate  the  effects  of  correlated  abilities  on  observed  test 
characteristics,  the  simulated  response  data  were  analyzed  using  item 
analysis  and  factor  analysis  techniques.  To  assess  the  effects  of 
correlated  abilities  on  the  use  of  the  multidimensional  model,  the 
parameters  of  the  model  were  estimated,  and  the  estimates  were  compared 
to  the  true  parameters. 

The  results  of  this  study  indicated  that  the  presence  of  correlated 
abilities  has  important  Implications  for  the  characteristics  of  test  data, 
and  for  the  application  of  multidimensional  item  response  theory  models. 

It  was  concluded  that  it  is  necessary  to  consider  latent  item  structure 
as  well  as  latent  ability  structure  in  test  construction  and  analysis. 

It  was  also  concluded  that  use  of  multidimensional  item  response  theory 
models  that  do  not  explicitly  account  for  correlated  abilities  may  result 
in  misinterpretation  of  the  underlying  dimensions.  It  was  suggested 
that  research  should  be  conducted  to  determine  the  nature  of  the 
misinterpretation  and  to  perhaps  develop  an  item  response  theory  analogue 
to  factor  rotation. 
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An  Investigation  of  the  Effect  of  Correlated  Abilities 
on  Observed  Test  Characteristics 


Because  of  the  required  assumption  of  unidimensionaii ty ,  much  of  the  item 
response  theory  (IRT)  methodology  that  has  been  developed  is  inappropriate  for 
a  wide  range  of  applications.  In  such  applications,  either  unidimensional 
sets  of  items  cannot  be  constructed,  or  they  are  not  desired.  Until  recently, 
in  such  circumstances  the  practitioner  has  been  forced  to  abandon  IRT  and 
adopt  more  traditional  test  analysis  procedures,  or  to  inappropriately  apply 
IRT  methods  and  hope  the  procedures  are  robust  to  violations  of  the 
unidimensionality  assumption.  Unfortunately,  such  robustness  has  not  been 
demonstrated. 

In  recent  years,  researchers  have  begun  grappling  with  the  dimensionality 
problem.  Several  IRT  models  have  been  proposed  for  the  multidimensional  case, 
and  recently  some  theory  and  procedures  have  been  developed  for  applying  such 
models  (Reckase  and  McKinley,  1982;  McKinley  and  Reckase,  1983a,  1983b).  The 
work  that  has  been  done  in  this  area  indicates  that  it  has  great  promise  for 
dealing  with  the  dimensionality  problem. 

In  multidimensional  item  response  theory  (MIRT),  one  of  the  most 
important  questions  that  has  not  yet  been  addressed  focuses  on  the  effect  of 
correlated  abilities  on  the  interpretation  of  model  parameters.  Logically,  it 
seems  desirable  to  construct  different,  homogeneous  (unidimensionai)  sets  of 
items  to  measure  each  ability  or  trait  of  interest.  In  the  case  of  unrelated 
abilities,  such  as  math  computation  ability  and  vocabulary  ability,  this  is  a 
practical  approach.  However,  if  the  abilities  of  Interest  are  related,  such 
as  in  the  case  of  reading  comprehension  and  vocabulary,  constructing  an  item 
set  that  measures  only  one  of  these  two  abilities  is  more  difficult. 

Developing  a  unidimensional  set  of  vocabulary  items  seems  easy,  but  how  does 
one  construct  reading  comprehension  items  that  do  not  also  include  at  least  a 
small  vocabulary  component? 

The  purpose  of  this  study  is  to  investigate  the  effects  of  varying 
degrees  of  correlation  between  abilities  on  observed  test  characteristics. 

This  research  has  two  primary  objectives.  The  first  objective  is  to  identify 
the  characteristics  of  response  data  yielded  in  the  case  of  correlated 
abilities.  If  unique  characteristics  can  be  identified  and  used  to 
distinguish  the  multidimensional  data  from  those  produced  in  the 
unidimensional  case,  then  it  should  be  possible  to  identify  real  test 
situations  in  which  a  MIRT  model  is  appropriate.  The  second  objective  of  this 
research  is  to  determine  the  effect  of  varying  degrees  of  correlation  between 
abilities  on  estimates  of  parameters  from  a  MIRT  model  which  does  not 
explicitly  account  for  such  a  correlation. 

Method 

MIRT  methodology  is  relatively  new  and  probably  unfamiliar  to  many. 
Therefore,  before  continuing  with  a  discussion  of  this  research,  a  brief 
discussion  of  the  MIRT  model  selected  for  this  study  will  be  presented.  For  a 
more  detailed  discussion  of  this  model,  see  McKinley  and  Reckase,  1983a. 
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The  MIRT  Model 


The  MIRT  model  selected  for  this  study  is  an  extension  of  the  two- 
parameter  logistic  (2PL)  model  proposed  by  Birnbaum  (1968).  The 
multidimensional  2PL  model,  or  M2PL  model,  is  given  by 


exp(d.  +  a. '  0 . ) 
_ i  —1—1 

1  +  exp(d^  + 


where  is  a  vector  of  ability  parameters  for  examinee  j ,  is  a  vector  of 
discrimination  parameters  for  Item  1,  Is  a  scalar  Item  parameter  related  to 
item  difficulty,  and  P£(_0_j)  is  the  probability  of  a  correct  response  to  item  1 
by  an  examinee  having  ability  . 

The  discrimination  and  ability  vectors  in  Equation  1  are  both  of  order  m, 
where  m  is  the  number  of  dimensions  comprising  the  complete  latent  space. 

The  a^'  ^  term  in  Equation  1  can  be  written  as 


^  ®jk  * 


where  is  the  item  discrimination  parameter  for  dimension  k  and  0^^^  is  the 

examinee  ability  parameter  for  dimension  k.  In  the  unidimensional  case 
Equation  1  simplifies  to  the  2PL  model  (without  the  D*1.7  term  usually  used  in 
the  2PL  model)  with 


where  bj^  is  the  difficulty  parameter  for  item  1  from  the  2PL  model. 
Design 


The  basic  design  of  this  study  involved  the  simulation  and  analysis  of 
response  data  generated  for  examinees  having  varying  levels  of  correlation 
between  their  abilities  on  different  dimensions.  The  generated  data  were  then 
analyzed  using  traditional  test  analysis  techniques  to  determine  the  effects 
of  the  correlated  abilities  on  the  test  characteristics.  Afterward,  the  data 
were  analyzed  using  the  M2PL  model  to  determine  the  effects  of  correlated 
abilities  on  the  estimates  of  the  parameters  of  the  model. 


Datasets 

Simulated  test  data  were  generated  for  two  different  types  of  test.  The 
first  test  type  measured  two  dimensions,  with  half  of  the  items  on  the  test 
measuring  predominantly  one  dimension,  and  other  half  measuring  predominantly 
the  second  dimension.  The  second  test  type  also  measured  two  dimensions,  but 
for  this  test  each  item  measured  both  dimensions.  Four  datasets  were 
generated  for  each  test  using  Interdlmenslonal  ability  correlations  of  .7,  .5 
.35,  and  0.  Table  1  summarizes  the  eight  datasets  that  were  created. 


Table  1 


Dataset 


Simulated  Datasets 


Test 


-  V-V.'  .- 


1 

1 

0.70 

2 

1 

0.50 

3 

1 

0.35 

4 

1 

0.00 

5 

2 

0.70 

6 

2 

0.50 

7 

2 

0.35 

8 

2 

0.00 

Table  2  shows  the  true  item  parameters  used  to  simulate  the  two  tests. 
Each  test  had  50  items,  and  the  same  set  of  d-parameters  was  used  for  both 
tests.  As  can  be  seen,  for  test  1  the  items  generally  have  high 
discriminations  on  one  dimension  or  the  other,  but  not  both.  For  test  2  the 
items  tend  to  have  high  discriminations  on  both  dimensions. 


Table  2 
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True  Item  Parameters  Used 
To  Simulate  Both  Tests 
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Table  2(Continued) 

True  Item  Parameters  Used 
To  Simulate  Both  Tests 


Item 

d 

Test  1 

Test  2 

®1 

®2 

^1 

32 

11 

-0.51 

1.38 

0.30 

0.98 

1.02 

12 

0.27 

1.39 

0.27 

0.93 

1.07 

13 

-0.14 

1.21 

0.73 

1.20 

0.75 

14 

-1.71 

1.30 

0.55 

0.68 

1.24 

15 

0.41 

1.39 

0.27 

0.68 

1.24 

16 

1.15 

1.37 

0.37 

1.02 

0.98 

17 

-0.52 

1.37 

0.35 

1.02 

0.98 

18 

0.40 

1.39 

0.28 

0.83 

1.15 

19 

-0.28 

1.41 

0.11 

1.25 

0.67 

20 

0.59 

1.34 

0.46 

1.06 

0.94 

21 

-2.77 

1.35 

0.41 

0.61 

1.27 

22 

-0.17 

1.36 

0.38 

0.90 

1.09 

23 

-0.58 

1.35 

0.44 

0.74 

1.21 

24 

-0.11 

1.39 

0.26 

0.72 

1.22 

25 

-0.61 

1.38 

0.30 

0.94 

1.06 

26 

1.20 

0.33 

1.38 

0.57 

1.30 

27 

0.14 

0.35 

1.37 

0.73 

1.21 

28 

1.40 

0.07 

1.41 

0.99 

1.01 

29 

0.34 

0.21 

1.40 

0.75 

1.20 

30 

-1.02 

0.32 

1.38 

0.86 

1.12 

31 

1.05 

0.34 

1.37 

0.83 

1.14 

32 

0.31 

0.55 

1.30 

0.74 

1.21 

33 

-0.83 

0.45 

1.34 

0.61 

1.27 

34 

0.55 

0.33 

1.37 

0.99 

1.01 

35 

1.46 

0.42 

1.35 

0.79 

1.17 

36 

1.05 

0.33 

1.38 

1.07 

0.92 

37 

-1.05 

0.37 

1.37 

1.11 

0.88 

38 

0.34 

0.54 

1.31 

0.80 

1.17 

39 

0.20 

0.37 

1.37 

0.68 

1.24 

40 

-0.45 

0.40 

1.36 

1.17 

0.79 

41 

1.67 

0.34 

1.37 

0.72 

1.22 

42 

-0.20 

0.27 

1.39 

0.93 

1.07 

43 

-0.69 

0.30 

1.38 

1.17 

0.79 

44 

0.35 

0.28 

1.39 

0.93 

1.07 

45 

0.93 

0.47 

1.34 

1.15 

0.82 

46 

0.86 

0.67 

1.25 

1.10 

0.89 

47 

-1.70 

0.41 

1.35 

1.14 

0.83 

48 

-1.54 

0.40 

1.36 

1.04 

0.95 

49 

-0.57 

0.33 

1.38 

1.06 

0.94 

50 

-0.78 

0.28 

1.39 

0.95 

1.05 

Mean 

-0.06 

0.86 

0.86 

0.89 

1.07 

S.D. 

1.04 

0.51 

0.51 

0.18 

0.16 

Each  group  of  examinees  consisted  of  2000  simulated  subjects  with  true 
abilities  selected  at  random  from  a  bivariate  normal  distribution  having  both 
means  equal  to  0.0,  both  standard  deviations  equal  to  (;.50,  and  having  the 
appropriate  correlation.  Each  group  was  used  for  only  one  of  the  two  tests. 
Thus,  there  were  eight  sets  of  item  responses  generated.  Each  set  of 
responses  was  stored  in  the  appropriate  dataset,  depending  on  the  test  and  the 
correlation  between  abilities.  The  same  set  of  analyses  were  then  run  on  each 
dataset. 

Analyses 

There  were  four  types  of  analyses  run  on  each  dataset  in  Table  1.  The 
first  type  was  an  item  analysis.  This  consisted  of  computing  item  proportion- 
correct  difficulty  indices,  item  point  blserial  discrimination  indices,  and  a 
KR-20  test  reliability  coefficient. 

The  second  type  of  analysis  performed  on  each  dataset  was  a  principal 
components  analysis.  More  specifically,  a  principal  components  analysis  of 
tetrachorlc  correlations  was  performed  on  each  dataset.  Using  the  results  of 
the  principal  components  analyses,  both  a  varimax  rotated  and  an  oblique 
rotated  factor  solution  were  obtained  for  each  dataset.  In  each  case  two 
factors  were  rotated. 

The  third  type  of  analysis  performed  was  the  application  of  the  M2PL 
model  to  the  data.  This  consisted  of  estimating  item  and  person  parameters 
for  the  M2PL  model  for  each  dataset.  Parameter  estimation  for  the  M2PL  model 
was  performed  using  the  MAXLOG  program  (McKinley  and  Reckase,  1983c). 

The  final  type  of  analysis  performed  on  these  data  consisted  of 
correlational  analyses.  Correlations  were  computed  among  true  and  estimated 
item  parameters,  item  statistics  (traditional  difficulty  and  discrimination), 
and  factor  loadings.  Correlations  between  true  and  estimated  ability 
parameters  were  also  computed. 

Results 

As  was  discussed  above,  there  were  four  types  of  analyses  performed  on 
the  eight  datasets  created  for  this  study  -  item  analyses,  principal  component 
analyses,  MIRT  analyses,  and  correlation  analyses.  The  results  obtained  from 
all  four  of  these  sets  of  analyses  will  be  presented  for  each  dataset 
separately,  beginning  with  the  four  datasets  based  on  the  first  test. 

Remember  that  the  first  test  contained  two  subsets  of  items,  each  of  which  was 
relatively  unidimensional.  After  presenting  these  results,  the  results  for 
the  second  test  will  be  presented.  The  second  test  had  items  that  each 
measured  two  dimensions. 


Test  1  Analyses 

Dataset  1.  Table  3  shows  the  results  of  the  item  analysis,  principal 
components  analysis,  and  MIRT  analysis  of  the  first  dataset.  This  dataset  was 
created  using  test  1  and  a  group  of  examinees  having  an  inter-dimension 
ability  correlation  of  0.70.  The  columns  headed  'Item  Parameter  Estimates' 
are  the  results  obtained  from  the  MIRT  analysis,  and  are  estimates  of  the  item 
parameters  of  the  M2PL  model.  The  columns  headed  'Item  Statistics'  are  the 
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proportion-correct  item  difficulties  (p)  and  item  point  biserial 
discrimination  indexes  (pbis)  obtained  from  the  item  analysis  of  the  first 
dataset.  Columns  6  and  7  are  the  varimax  rotated  factor  loadings  for  the 
first  two  factors  of  the  principal  components  analysis  of  tetrachoric 
correlations.  The  last  two  columns  are  oblique  rotated  factor  loadings  for 
the  first  two  factors  from  the  principal  components  analysis. 


Table  3 

Item  Parameter  Estimates,  Item  Statistics,  and  Factor 
Loadings  for  Dataset  1 


Item  Parameter  Item 

Factor 

Loadings 

Estimates  Statistics 

Orthogonal 

Oblique 

aj 

.43 

0.88 

.69 

0.73 

.72 

0.89 

.80 

0.44 

.46 

0.91 

.64 

0.59 

.72 

0.77 

.51 

0.74 

.38 

1.02 

.43 

0.97 

.41 

1.11 

.45 

0.68 

.59 

1.00 

.66 

0.78 

.46 

0.73 

.46 

0.91 

.45 

1.03 

.45 

1.02 

.06 

1.33 

.60 

0.81 

.37 

0.73 

.51 

0.89 

.51 

0.95 

.60 

0.67 

.0 

2.00 

.01 

0.48 

.88 

0.4 

6 

.78 

0.2 

8 

.92 

0.3 

4 

.95 

0.4 

0 

.62 

0.2 

6 

.70 

0.4 

4 

.70 

.34 

0.5 

0.2 

0 

9 
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Item  Parameter 


Estimates 


Statistics 


Factor  Loadings 
Orthogonal  Oblique 


The  mean  score  on  test  1  for  this  group  of  examinees  was  24,14,  and  the 
standard  deviation  was  8.13.  The  KR-20  reliability  for  these  data  was  0.86. 
The  correlation  between  the  factors,  obtained  from  the  oblique  solution,  was 
0.64. 

Table  4  shows  the  intercorrelation  matrix  for  the  true  and  estimated  item 
parameters  for  the  first  dataset.  As  can  be  seen,  the  correlations  of  the 
true  and  estimated  item  parameters  were  0.996  for  the  d-parameter,  0.731  for 
the  true  a  on  the  first  dimension  and  the  estimated  a  for  the  second 
dimension,  and  0.768  for  the  true  a  for  the  second  dimension  and  the  estimated 
a  for  the  first  dimension.  Thus,  the  d-parameter  was  very  well  estimated,  and 
the  a-parameters  were  only  moderately  well  estimated. 


"  »  *  K  *  •  ' 


■*.  %  , 


Variable 


True 


Estimated 


Table  5  shows  the  Intercorrelatlon  matrix  of  the  true  and  estimated 
ability  parameters  obtained  for  the  first  dataset.  As  can  be  seen,  the 
ability  on  dimension  1  had  a  correlation  of  0.670  with  the  ability  estimate  on 
the  second  dimension,  while  there  was  a  correlation  of  0.704  between  the  true 
ability  for  dimension  2  and  the  ability  estimate  on  dimension  1.  Despite  the 
correlation  of  0.685  obtained  for  the  true  abilities,  the  estimated  abilities 
were  not  correlated  (r=-0.140).  Thus,  while  the  abilities  for  this  group  were 
moderately  well  estimated,  the  correlation  between  the  dimensions  was  not 
recovered  by  the  estimation  process. 


Table  5 

Intercorrelatlon  Matrix  for  True  and  Estimated 
Ability  Parameters  for  Dataset  1 


1.000 


0.685 

1.000 


0.444 

0.704 

1.000 


0.67 

0.39 

-0.14 

1.00 


The  correlation  of  the  proportion-correct  difficulty  Index  and  the  d- 
parameter  was  0.995  (for  true  and  estimated  d-values),  which  is  about  what  was 
expected.  The  point  blserlal  discrimination  index  had  a  correlation  of  -0.131 
with  the  true  a-parameter  for  dimension  1  and  0.166  for  the  second 
dimension.  Using  the  a-value  estimates  the  correlation  was  0.258  for 
dimension  1  and  0.059  for  dimension  2.  This,  too,  was  much  as  was  expected. 
Since  the  point  biserial  is  strongly  affected  by  the  dimensionality  of  the 


Items,  It  should  not  have  a  strong  relationship  to  discrimination  on  a  single 
dimension  for  two-dimensional  data. 


Table  6  shows  the  Intercorrelatlon  matrix  for  the  true  and  estimated  item 
parameters  and  the  varlmax  and  oblique  rotated  factor  loadings  for  the  first 
dataset.  As  can  be  seen,  there  was  a  strong  relationship  between  the  factor 
loadings  (both  varlmax  and  oblique  rotated)  and  the  Item  parameters  (both  true 
and  estimated).  The  first  four  eigenvalues  obtained  from  the  principal 
components  analysis  of  these  data  were  10.01,  1.50,  1.31,  and  1.27.  There 
appeared  to  be  a  strong  first  factor  and  a  much  smaller  second  factor.  This 
Is  consistent  with  the  high  Inter-dlmenslon  ability  correlation  for  these 
data. 


Table  6 

Intercorrelatlon  Matrix  for  True  and  Estimated  Item 
Parameters  and  Factor  Loadings  for  Dataset  1 


Variable 


Item  Parameters 


Estimated 


Factor  Loadings 
Orthogonal  Oblique 


True 

d  1.000 

-0.172 

0. 

.159 

0. 

.996 

0. 

.317 

-0. 

.272 

0. 

.330 

-0. 

,308 

0.329 

-0.319 

1.000 

-0, 

.987 

-0. 

.189 

-0. 

.751 

0, 

.731 

-0, 

.852 

0. 

.860 

-0.868 

0.872 

^2 

1, 

.000 

0. 

.177 

0. 

.768 

-0, 

.730 

0, 

.862 

-0, 

.842 

0.871 

-0.862 

Estimated 

d^ 

1, 

.000 

0, 

.358 

-0, 

.323 

0, 

.362 

-0, 

.341 

0.362 

-0.352 

1, 

.000 

-0, 

.841 

0, 

.953 

-0, 

.852 

0.942 

-0.894 

^2 

1, 

.000 

-0, 

.852 

0. 

.90S 

-0.880 

0.908 

Orthogonal 

I 

1, 

.000 

-0. 

,918 

0.995 

-0.956 

II 

1, 

.000 

-0.954 

0.994 

Oblique 

I 

1.000 

-0.981 

The  correlation  of  the  point  blserlal  Index  and  the  factor  loadings  was 
0.269  and  0.091  for  the  two  varlmax  rotated  factors,  and  0.180  and  -0.009  for 
the  two  oblique  rotated  factors.  The  proportion-correct  difficulty  Index  had 
correlations  of  0.329  and  -0.323  with  the  varlmax  rotated  loadings,  and  0.332 
and  -0.330  with  the  oblique  rotated  loadings.  The  proportion-correct  and 
point  blserlal  indexes  had  a  correlation  of  0.086. 

Dataset  2.  Table  7  shows  the  Item  parameter  estimates.  Item  statistics, 
and  factor  loadings  obtained  for  dataset  2.  These  data  were  generated  using 
test  1  and  a  group  of  examinees  with  an  inter-dimension  ability  correlation  of 
0.50.  The  mean  score  on  test  1  for  this  group  was  24.49,  and  the  standard 
deviation  was  7.73.  The  KR-20  reliability  was  0.84,  which  Is  slightly  lower 
than  the  KR-20  for  dataset  1.  The  correlation  between  the  factors,  obtained 
from  the  oblique  rotation,  was  -0.59,  which  is  slightly  lower  than  for  dataset 
1,  and  opposite  In  sign. 
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Table  7 


Item  Parameter  Estitr  tea,  Item  Statistics,  and  Factor 
Loadings  for  Dataset  2 


1% 


*  *• 

fc.w: 


i 
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w. 
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fe 


r.: 


Item 

Item  Parameter 

Estimates 

Item 

Statistics 

Factor  Loadings 

Orthogonal  Oblique 

d 

«1 

32 

P 

pbls 

I 

II 

I 

II 

1 

-1.41 

0.55 

0.91 

0.23 

0.28 

0.23 

0.41 

0.11 

-0.40 

2 

2.23 

0.62 

0.80 

0.87 

0.24 

0.27 

0.36 

0.18 

-0.32 

3 

-0.05 

0.29 

1.07 

0.49 

0.29 

0.10 

0.50 

-0.07 

-0.55 

4 

2.15 

0.13 

1.19 

0.85 

0.22 

0.06 

0.51 

-0.12 

-0.57 

5 

-0.81 

0.37 

1.06 

0.34 

0.29 

0.14 

0.48 

-0.02 

-0.50 

6 

-1.31 

0.33 

0.83 

0.24 

0.24 

0.12 

0.41 

-0.01 

-0.43 

7 

0.49 

0.61 

0.75 

0.60 

0.31 

0.30 

0.35 

0.22 

-0.30 

8 

-0.47 

0.42 

0.96 

0.40 

0.30 

0.19 

0.43 

0.06 

-0.43 

9 

-0.52 

0.46 

0.69 

0.39 

0.27 

0.19 

0.37 

0.08 

-0.37 

10 

-1.69 

0.44 

1.16 

0.21 

0.28 

0.16 

0.48 

0.01 

-0.50 

11 

-0.57 

0.32 

0.96 

0.38 

0.28 

0.12 

0.47 

-0.04 

-0.50 

12 

0.30 

0.41 

1.03 

0.56 

0.31 

0.18 

0.46 

0.04 

-0.47 

13 

-0.12 

0.72 

0.70 

0.47 

0.33 

0.33 

0.34 

0.26 

-0.27 

14 

-1.89 

0.65 

1.02 

0.18 

0.29 

0.26 

0.42 

0.14 

-0.40 

15 

0.41 

0.53 

0.90 

0.59 

0.32 

0.23 

0.43 

0.10 

-0.42 

16 

1.22 

0.49 

0.85 

0.74 

0.28 

0.22 

0.40 

0.11 

-0.39 

17 

-0.56 

0.43 

0.84 

0.38 

0.29 

0.20 

0.39 

0.09 

-0.38 

18 

0.35 

0.49 

0.83 

0.57 

0.31 

0.21 

0.41 

0.10 

-0.40 

19 

-0.40 

0.0 

1.86 

0.44 

0.27 

0.02 

0.54 

-0.19 

-0.63 

20 

0.68 

0.66 

0.95 

0.64 

0.36 

0.29 

0.44 

0.17 

-0.41 

21 

-2.74 

0.31 

0.94 

0.08 

0.17 

0.10 

0.41 

-0.04 

-0.45 

22 

-0.19 

0.42 

0.95 

0.46 

0.31 

0.19 

0.43 

0.06 

-0.44 

23 

-0.63 

0.56 

0.80 

0.37 

0.31 

0.22 

0.42 

0.10 

-0.41 

24 

-0.20 

0.39 

0.92 

0.46 

0.30 

0.18 

0.43 

0.06 

-0.43 

25 

-0.72 

0.40 

0.81 

0.35 

0.27 

0.18 

0.40 

0.06 

-0.40 

26 

1.29 

0.80 

0.27 

0.76 

0.24 

0.37 

0.16 

0.38 

-0.04 

27 

0.18 

0.80 

0.55 

0.54 

0.32 

0.38 

0.27 

0.35 

-0.17 

28 

1.47 

0.71 

0.28 

0.79 

0.21 

0.37 

0.10 

0.40 

29 

0.40 

0.93 

0.23 

0.58 

0.27 

0.43 

0.12 

0.46 

30 

-1.02 

0.84 

0.45 

0.29 

0.28 

0.40 

0.20 

0.39 

-0.08 

31 

1.31 

1.32 

0.25 

0.73 

0.30 

0.53 

0.11 

0.59 

0.08 

32 

0.28 

1.00 

0.50 

0.56 

0.33 

0.46 

0.21 

0.46 

-0.07 

33 

-0.79 

0.88 

0.29 

0.34 

0.26 

0.43 

0.12 

0.46 

34 

0.63 

1.11 

0.28 

0.63 

0.30 

0.49 

0.12 

0.53 

0.05 

35 

1.46 

0.77 

0.42 

0.78 

0.25 

0.38 

0.19 

0.38 

-0.08 

36 

1.05 

0.79 

0.29 

0.72 

0.24 

0.40 

0.12 

0.43 

0.01 

37 

-1.21 

1.23 

0.18 

0.28 

0.27 

0.50 

0.08 

0.56 

0.10 

38 

0.43 

1.14 

0.23 

0.59 

0.29 

0.50 

0.10 

0.55 

0.07 

39 

0.19 

0.93 

0.42 

0.54 

0.32 

0.42 

0.22 

0.42 

-0.09 

40 

-0.53 

0.92 

0.30 

0.39 

0.28 

0.44 

0.14 

0.46 

0.01 

41 

1.69 

0.73 

0.40 

0.82 

0.23 

0.36 

0.18 

0.35 

-0.08 

42 

-0.22 

0.78 

0.36 

0.45 

0.28 

0.39 

0.18 

0.39 

-0.06 
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Table  7(Contlnued) 


Item  Parameter  Estimates,  Item  Statistics,  and  Factor 
Loadings  for  Dataset  2 


item  Parameter 
Item  Estimates 


Item 

Statistics 


Factor  Loadings 


Orthogonal  Oblique 


d 

^1 

32 

P 

phis 

I 

II 

I 

II 

43 

-0.74 

0.79 

0.58 

0.35 

0.31 

0.36 

0.28 

0.32 

-0.19 

44 

0.34 

0.72 

0.27 

0.58 

0.25 

0.36 

0.15 

0.36 

-0.04 

45 

0.93 

1.16 

0.35 

0.68 

0.32 

0.49 

0.16 

0.52 

-0.00 

46 

0.88 

0.90 

0.51 

0.68 

0.31 

0.42 

0.23 

0.41 

-0.11 

47 

-1.74 

0.90 

0.46 

0.18 

0.25 

0.38 

0.21 

0.37 

-0.10 

48 

-1.76 

1.28 

0.27 

0.20 

0.27 

0.48 

0.14 

0.52 

0.03 

49 

-0.55 

0.69 

0.39 

0.38 

0.26 

0.35 

0.19 

0.34 

-0.09 

50 

-0.77 

0.77 

0.40 

0.34 

0.27 

0.39 

0.18 

0.39 

-0.07 

Table  8 

shows  the 

Intercorrelation 

matrix 

for  the 

true  and 

estimated 

item 

parameters  for  dataset  2.  The  true  and  estimated  d-parameters  had  a 
correlation  of  0.998,  indicating  that  the  d-parameter  was  once  again  very  well 
estimated.  The  correlation  between  the  true  and  estimated  a-values  was  0.866 
for  the  dimension  1  true  value  and  the  dimension  2  estimated  value,  and  0.834 
for  the  dimension  2  true  value  and  dimension  1  estimated  value.  The  a-values, 
then,  were  better  estimated  for  dataset  2  than  for  dataset  1. 


Table  8 


Intercorrelation  Matrix  for  True  and  Estimated 
Item  Parameters  for  Dataset  2 


True 

Estimated 

Variable 

d 

32 

d 

®1 

32 

True 

d 

1.000 

-0.172 

0.159 

0.998 

0.112 

-0.182 

®1 

32 

d 

1.000 

-0.987 

-0.180 

-0.784 

0.866 

1.000 

0.167 

0.834 

-0.886 

Estimated 

1.000 

0.115 

-0.189 

®1 

32 

1.000 

-0.865 

1 .000 

Table  9  shows  the  Intercorrelation  matrix  of  the  true  and  estimated 
ability  parameters  obtained  for  the  second  dataset.  The  true  ability  on 
dimension  1  had  a  correlation  of  0.716  with  the  estimated  ability  on  dimension 
2,  while  there  was  a  correlation  of  0.743  for  the  true  ability  for  dimension  2 
and  the  dimension  1  estimated  ability.  The  inter-dimension  ability 
correlation  was  0.494  for  the  true  values,  and  -0.150  for  the  estimated 


values.  These  ability  parameters  were  estimated  slightly  better  than  were  the 
parameters  for  dataset  1,  but  once  again  the  Inter-dlmenslon  ability 
correlation  was  not  recovered  during  the  estimation  process. 

Table  9 


Intercorrelation  Matrix  for  True  and  Estimated 
Ability  Parameters  for  Dataset  2 


True 

Estimated 

Variable 

®1 

®2 

®1 

®2 

True 

1.000 

0.494 

0.321 

0.716 

1.000 

0.743 

-0.279 

Estimated 

®2 

1.000 

-0.150 

1.000 

The  correlation  of  the  proportion-correct  difficulty  index  and  the  d- 
parameter  was  0.993  (0.995  for  the  estimated  d-parameter) .  The  point  biserlal 
index  had  a  correlation  with  the  true  a-parameters  of  0.187  for  the  first 
dimension  and  -0.134  for  dimension  2.  Those  correlations  were  0.166  and  0.123 
when  computed  with  the  estimated  a-values. 

Table  10  shows  the  intercorrelation  matrix  for  the  true  and  estimated 
item  parameters  and  the  factor  loadings  for  dataset  2.  Again,  the 
relationship  between  the  true  and  estimated  item  parameters  and  the  varlmax 
and  oblique  rotated  factor  loadings  were  quite  strong.  The  first  four 
eigenvalues  from  the  principal  components  analysis  were  9.09,  1.79,  1.30,  and 
1.28.  The  first  factor  for  these  data  was  slightly  smaller  than  for  the  first 
dataset,  and  the  second  factor  slightly  larger.  This  is  consistent  with  the 
lower  inter-dlmenslon  ability  correlation  for  this  group  of  examinees. 


Table  10 


Intercorrelation  Matrix  for  True  and  Estimated  Item 
Parameters  and  Factor  Loadings  for  Dataset  2 


Variable 

Item  Parameters 

Factor  ! 

Loadings 

True 

Estimated 

Orthogonal 

Oblique 

d 

^1 

^2 

d 

®1 

®2 

I  II 

I 

II 

True 

d  1 

.000 

-0.172 

0.159 

0.998 

0.112 

-0.182 

0.201  -0.191 

0.202 

0.196 

1.000 

-0.987 

-0.180 

-0.784 

0.866 

-0.859  0.936 

-0.890 

-0.930 

ao 

1.000 

0.167 

0.834 

-0.886 

0.899  -0.943 

0.923 

0.945 

Estimated 

d^ 

1.000 

0.115 

-0.189 

0.204  -0.197 

0.205 

0.201 

Si 

1.000 

-0.865 

0.976  -0.876 

0.963 

0.911 

aj 

1.000 

-0.907  0.954 

-0.931 

-0.955 

Orthogonal 

I 

1.000  -0.930 

0.996 

0.959 

11 

1.000 

-0.961 

-0.996 

Oblique 

I 

1.000 

0.981 

11 

1.000 

The  correlation  of  the  point  blserial  index  and  the  factor  loadings  was 
0.138  and  0.185  for  the  varlmax  rotated  loadings  and  0.057  and  -0.109  for  the 
oblique  rotated  loadings.  The  proportion-correct  difficulty  index  had 
correlations  of  0.212  and  -0.208  with  the  varimax  rotated  loadings  and  0.214 
and  0.212  with  the  oblique  rotated  loadings.  The  proportion-correct 
difficulty  index  and  the  point  biserlal  index  had  a  correlation  of  0.008. 

Dataset  3.  Table  11  shows  the  item  parameter  estimates,  item  statistics, 
and  factor  loadings  for  dataset  3.  These  data  were  generated  using  test  1  and 
a  group  of  examinees  with  an  inter-dimension  ability  correlation  of  0.35.  The 
mean  score  on  test  1  for  this  group  was  24.66  and  the  standard  deviation  was 
7.25.  The  KR-20  reliability  was  0.82.  This  was  slightly  lower  than  for 
dataset  2.  The  correlation  between  the  factors,  obtained  from  the  oblique 
rotation,  was  0.52,  which  is  slightly  lower  than  for  dataset  2,  and  opposite 
in  sign. 
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Item  Parameter 


Estimates 


illll 


Statistics 


Factor  Loadings 
Orthogonal  Oblique 


p 

pbis 

.23 

0.22 

.90 

0.16 

.51 

0.27 

.85 

0.23 

.33 

0.24 

.24 

0.26 

.63 

0.29 

.42 

0.24 

.40 

0.25 

.18 

0.24 

.39 

0.27 

.55 

0.29 

.47 

0.31 

.19 

0.22 

.59 

0.25 

,75 

0.25 

.38 

0.26 

.58 

0.23 

.44 

0.24 

.63 

0.27 

,07 

0.17 

.46 

0.27 

.39 

0.28 

.48 

0.25 

.39 

0.29 

.75 

0.23 

.54 

0.26 

.78 

0.20 

.56 

0.28 

.29 

0.26 

.71 

0.25 

.57 

0.31 

.33 

0.30 

.62 

0.28 

.79 

0.21 

34 

0 

37 

-0 

37 

0 

49 

-0 

40 

0 

Table  1 1 (Continued) 


Item  Parameter  Estimates,  Item  Statistics,  and  Factor 
Loadings  for  Dataset  3 


Item 


Item  Parameter 
Estimates 


Item 

Statistics 


Factor  Loadings 


Orthogonal  Oblique 


d 

^1 

^2 

P 

phis 

I 

II 

I 

II 

36 

1.05 

0.80 

0.34 

0.71 

0.55 

0.39 

0.16 

0.39 

0.06 

37 

-1.08 

0.88 

0.34 

0.28 

0.26 

0.41 

0.17 

0.40 

0.07 

38 

0.39 

0.96 

0.34 

0.58 

0.30 

0.45 

0.17 

0.45 

-0.00 

39 

0.16 

0.90 

0.24 

0.53 

0.27 

0.43 

0.12 

0.45 

-0.00 

40 

-0.46 

0.89 

0.26 

0.40 

0.27 

0.44 

0.12 

0.45 

0.00 

41 

1.83 

1.08 

0.14 

0.82 

0.22 

0.47 

0.06 

0.51 

-0.08 

42 

-0.19 

0.97 

0.18 

0.46 

0.26 

0.45 

0.09 

0.48 

-0.03 

43 

-0.68 

0.73 

0.33 

0.35 

0.25 

0.38 

0.15 

0.38 

0.05 

44 

0.37 

0.75 

0.30 

0.58 

0.26 

0.37 

0.17 

0.37 

0.07 

45 

1.04 

0.80 

0.32 

0.71 

0.25 

0.40 

0.15 

0.41 

0.05 

46 

0.92 

0.71 

0.54 

0.69 

0.28 

0.34 

0.26 

0.30 

0.19 

47 

-1.68 

0.86 

0.39 

0.19 

0.24 

0.39 

0.18 

0.38 

0.08 

48 

-1.59 

0.80 

0.41 

0.20 

0.24 

0.38 

0.19 

0.37 

0.10 

49 

-0.58 

0.98 

0.39 

0.38 

0.30 

0.45 

0.18 

0.45 

0.06 

-0.74 

0.74 

0.54 

0.34 

0.29 

0.37 

0.26 

0.33 

0.18 

Table  12  shows  the  Intercorrelation  matrix  for  the  true  and  estimated 
item  parameters  for  dataset  3.  The  true  and  estimated  d-parameters  had  a 
correlation  of  0.999,  indicating  that  the  d-parameter  was  very  well 
estimated.  The  dimension  1  true  a-values  had  a  correlation  of  0.921  with  the 
dimension  2  estimated  a-values,  while  there  was  a  correlation  of  0.937  between 
the  dimension  2  true  a-values  and  the  dimension  1  estimated  a-values.  The  a- 
values,  then,  were  fairly  well  estimated  for  these  data. 


Variable 


Table  13  shows  the  intercorrelation  matrix  of  the  true  and  estimated 
ability  parameters  obtained  for  dataset  3.  The  true  abilities  for  dimension  1 
had  a  correlation  of  0.772  with  dimension  2  of  the  estimated  abilities.  The 
dimension  2  true  abilities  had  a  correlation  of  0.779  with  the  dimension  1 
estimated  abilities.  The  inter-dimension  ability  correlation  was  0.345  for 
the  true  abilities  and  -0.087  for  the  estimated  abilities. 


Table  13 

Inter correlation  Matrix  for  True  and  Estimated 
Ability  Parameters  for  Dataset  3 


Variable 


True 


Estimated 


True  9 

0 

Estimated  6 
0 


1 


1 


1.000 


0.345 

1.000 


0.231 

0.779 

1.000 


0.772 

0.229 

-0.087 

1.000 


The  correlation  between  the  d-pararaeter  and  the  proportion-correct 
difficulty  index  was  0.995  (0.993  for  the  d-parameter  estimates).  The 
correlation  of  the  point  blserlal  discrimination  index  with  the  true  a- 
parameters  was  -0.134  for  dimension  1  and  0.171  for  dimension  2.  When 
estimated  a-values  were  used,  these  correlations  were  0.261  and  -0.079. 

Table  14  shows  the  Intercorrelation  matrix  for  the  true  and  estimated 
item  parameters  and  the  rotated  factor  loadings  for  dataset  3.  As  was  the 
case  previously,  there  was  a  strong  relationship  between  the  true  and 
estimated  item  parameters  and  both  sets  of  rotated  loadings.  The  first  four 
eigenvalues  from  the  principal  components  analysis  of  these  data  were  7.92, 
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2,08,  1,43,  and  1,29,  The  first  factor  for  these  data  was  smaller  than  for 
the  previous  datasets,  and  the  second  factor  was  somewhat  larger. 


Table  14 


Intercorrelation  Matrix  for  True  and  Estimated  Item 
Parameters  and  Factor  Loadings  for  Dataset  3 


Item  Parameter  Factor  Loadings 


Variable  True  Estimated  Orthogonal  Oblique 


d 

®1  ^2 

d 

®1  ^2 

I 

II 

I 

II 

True 

d  1 

.000 

-0.172  0.159 

0.999 

0.131  -0.207 

0.149 

-0.215 

0.164 

-0.204 

^1 

1.000  -0.987 

-0.164 

-0.929  0.921 

-0.946 

0.949 

-0.953 

0.955 

®2 

1.000 

0.153 

0.937  -0.915 

0.959 

-0.942 

0.961 

-0.952 

Estimated 

d" 

1.000 

0.127  -0.201 

0.145 

-0.200 

0.160 

-0.199 

1.000  -0.936 

0.988 

-0.949 

0.987 

-0.965 

a-, 

1.000 

-0.934 

0.989 

-0.957 

0.984 

Orthogonal 

I 

1.000 

-0.954 

0.997 

-0.972 

11 

1.000 

-0.972 

0.997 

Oblique 

I 

1.000 

-0.986 

II 

1.000 

The  correlation  of  the  point  blserlal  index  and  the  factor  loadings  was 
0,299  and  -0.050  for  the  varimax  rotation,  and  0.247  and  -0,106  for  the 
oblique  rotation.  The  proportion-correct  difficulty  Index  had  correlations  of 
0.147  and  -0.214  with  the  varimax  rotated  loadings,  and  0.162  and  -0.203  with 
the  oblique  rotated  loadings.  The  proportion-correct  difficulty  and  point 
blserlal  discrimination  Indexes  had  a  correlation  of  -0.101, 

Dataset  4.  Table  15  shows  the  Item  parameter  estimates,  Item  statistics, 
and  factor  loadings  for  dataset  4.  These  data  were  generated  using  test  1  and 
a  group  of  examinees  with  an  Inter-dlmension  ability  correlation  of  0.00.  The 
mean  score  on  test  1  for  this  group  was  24,61  and  the  standard  deviation  was 
6.52.  The  KR-20  reliability  was  0.77,  which  Is  somewhat  lower  than  for 
dataset  3.  The  correlation  between  the  factors,  obtained  from  the  oblique 
rotation,  was  0.36,  which  Is  slightly  lower  than  was  the  case  for  dataset  3. 


Item  Parameter 


Item 


Factor  Loadings 


Estimates 


Statistics 


Orthogonal 


Oblique 


p 

pbls 

.23 

0.20 

.90 

0.13 

.51 

0.23 

.85 

0.18 

.33 

0.21 

.24 

0.23 

.63 

0.24 

.41 

0.20 

.39 

0.20 

.17 

0.21 

.39 

0.24 

.55 

0.24 

.47 

0.26 

.18 

0.18 

.59 

0.20 

.75 

0.21 

.38 

0.22 

.59 

0.19 

.44 

0.21 

.63 

0.23 

.07 

0.15 

.46 

0.24 

.39 

0.23 

.48 

0.21 

.39 

0.23 

.75 

0.18 

.54 

0.22 

.78 

0.17 

.56 

0.25 

.29 

0.23 

.72 

0.21 

.57 

0.26 

.32 

0.28 

.62 

0.23 

.80 

0.18 

15 

0.33 

0.10 

04 

0.35 

-0.03 

11 

0.39 

0.04 

03 

0.47 

-0.06 

08 

0.40 

O.Oi 

12 

0.43 

0.04 

09 

0.46 

0.01 

04 

0.41 

-0.04 

02 

0.45 

-0.07 

08 

0.48 

-0.01 

07 

0.46 

-0.02 

05 

0.48 

-0.04 

25 

0.32 

0.20 

13 

0.33 

0.07 

06 

0.40 

-0.01 

12 

0.39 

0.04 

10 

0.40 

0.02 

03 

0.41 

-0.05 

01 

0.48 

-0.10 

12 

0.40 

0.04 

08 

0.40 

0.00 

17 

0.36 

0.11 

12 

0.40 

0.05 

09 

0.37 

0.02 

12 

0.40 

0.04 

43 

-0.00 

0.45 

39 

0.09 

0.39 

42 

-0.00 

0.44 

48 

0.06 

0.49 

47 

0.05 

0.49 

38 

0.09 

0.38 

39 

0.17 

0.38 

46 

0.16 

0.45 

44 

0.06 

0.46 

94 

69 


1.000 

0.007 

0.0 

1.000 

0.8 

l.Oi 

The  proportion-correct  difficulty  index  had  a  correlation  of  0.995  with 
the  true  d-parameter,  and  a  correlation  of  0.993  with  the  d-parameter 
estimates.  The  correlation  between  the  point  biserial  index  and  the  true  a- 
parameters  was  -0.1A6  for  dimension  1  and  0.183  for  dimension  2.  These  values 
were  0.274  and  -0.097  for  the  estimated  a  values. 

Table  18  shows  the  Intercorrelation  matrix  for  the  true  and  estimated 
item  parameters  and  the  rotated  factor  loadings  for  dataset  4.  As  has  been 
the  case  all  along,  there  was  a  strong  relationship  between  the  item 
parameters  and  estimates  and  both  sets  of  rotated  factor  loadings.  The  first 


four  eigenvalues  from  the  principal  components  analysis  for  dataset  4  were 
6.32,  2.74,  1.42,  and  1.32. 


Table  18 


Intercorrelation  Matrix  for  True  and  Estimated  Item 
Parameters  and  Factor  Loadings  for  Dataset  4 


Item  Parameters 

Factor 

Loadings 

Variable 

True 

Estimated 

Orthogonal 

Oblique 

d 

®1 

®2 

d 

®1 

&2 

I 

II 

I  II 

True 

d  1 

.000 

-0.172 

0.159 

0.999 

0.133 

-0.205 

0.\hl 

-0.217 

0.163  -0.203 

^1 

1.000 

-0.987 

-0.165 

-0.929 

0.920 

-0.945 

0.949  ■ 

-0.953  0.976 

®2 

1.000 

0.153 

0.937 

-0.914 

0.958 

-0.942 

0.962  -0.953 

Estimated 

d 

1.000 

0.129 

-0.199 

0.144 

-0.211 

0.159  -0.198 

1.000 

-0.937 

0.988 

-0.949 

0.987  -0.965 

^2 

1 .000 

-0.939 

0.989  ■ 

-0.957  0.985 

Orthogonal 

r 

1.000 

-0.955 

0.998  -0.973 

II 

1.000  ■ 

-0.972  0.998 

Oblique 

I 

1.000  -0.986 

II 

1.000 

The  point  biserial  index  had  correlations  of  0,312  and  -0.066  with  the 
varlmax  rotated  factor  loadings.  The  correlations  between  the  point  blserlals 
and  the  oblique  rotated  factor  loadings  were  0.262  and  -0.121.  The 
correlations  between  the  proportion-correct  index  and  the  varlmax  rotated 
factor  loadings  were  0.146  and  -0.217,  while  correlations  of  0.163  and  -0.203 
were  obtained  between  the  point  biserials  and  the  oblique  rotated  factor 
loadings.  The  correlation  between  the  proportion-correct  difficulty  index  and 
the  point  blserlal  discrimination  index  was  -0.110, 

Test  2  Analyses 

Dataset  5.  Table  19  shows  the  item  parameter  estimates,  item  statistics, 
and  rotated  factor  loadings  for  dataset  5.  These  data  were  generated  using 
test  2  and  a  group  of  examinees  having  an  inter-dimension  ability  correlation 
of  0.70.  The  mean  score  on  test  2  for  these  examinees  was  24.19,  while  the 
standard  deviation  was  9.00.  The  KR-20  reliability  was  0.89.  The  correlation 
between  factors,  obtained  from  the  oblique  rotation,  was  0.62. 
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Table  19 


Item  Parameter  Estimates,  Item  Statistics,  and  Factor 
Loadings  for  Dataset  5 


Item  Parameter 
Estimates 


Factor  Loadings 


Statistics  Orthogonal 


Oblique 


Item  Parameter 


Item 


Factor  Loadings 


Item 


Estimates  Statistics  Orthogonal  Oblique 


d 

^1 

®2 

P 

pbls 

I 

II 

I 

II 

39 

0.19 

0.89 

0.57 

0.54 

0.35 

0.45 

0.22 

0.50 

0.01 

40 

-0.49 

1.03 

0.60 

0.40 

0.38 

0.49 

0.23 

0.55 

-0.01 

41 

1.70 

1.16 

0.48 

0.79 

0.32 

0.50 

0.19 

0.57 

-0.06 

42 

-0.36 

0.07 

0.45 

0.39 

0.57 

0.15 

0.67 

-0.14 

43 

-0.91 

0.94 

0.74 

0.32 

0.37 

0.47 

0.28 

0.50 

0.06 

44 

0.32 

0.99 

0.56 

0.57 

0.37 

0.44 

0.26 

0.48 

0.05 

45 

0.91 

0.98 

0.54 

0.68 

0.35 

0.49 

0.20 

0.55 

-0.04 

46 

0.94 

0.51 

1.16 

0.68 

0.33 

0.20 

0.50 

0.11 

0.46 

47 

-1.86 

0.77 

0.85 

0.17 

0.31 

0.29 

0.42 

0.24 

0.32 

-1.60 

0.93 

0.49 

0.20 

0.29 

0.44 

0.19 

0.49 

-0.03 

49 

0.77 

0.84 

0.38 

0.37 

0.34 

0.39 

0.31 

0.26 

50 

-0.84 

0.76 

0.99 

0.34 

0.37 

0.32 

0.43 

0.28 

0.31 

Table  20  shows  the 

intercorrelation  matrix  obtained  for  the  true 

and 

estimated  Item  parameters  for  dataset  5 

.  The 

correlation  between  the 

true  and 

estimated  d-parameters 

was  0.997 

The 

correlations  of 

the  true 

and  estimated 

a-parameters  for 

these 

data  were 

0.198 

for  dimension  1 

and  0.167  for  dimension 

2. 

These  values 

are  In 

marked  contrast 

to  the 

high  values  obtained  for  test 

1. 

Here,  there 

is  no  significant  correlation 

between 

the  true 

and  estimated 

Item  discrimination  parameters. 

Table  20 

Intercorrelation  Matrix  for  True  and  Estimated 

Item  Parameters  for  Dataset  5 

True 

Variable  _ 

d 

®1 

32 

d 

®1 

32 

True  d 

1.000 

-0.048 

0.076 

0.997 

0.029 

0.088 

®1 

1.000 

-0.985 

-0.036 

0.198 

-0.176 

®2 

1.000 

0.067 

-0.159 

0.167 

Estimated  d 

1.000 

0.012 

0.096 

®l 

1.000 

-0.837 

®2 

1.000 

Table  21  shows  the  intercorrelation  matrix  for  the  true  and  estimated 
ability  parameters  for  dataset  5.  As  can  be  seen,  the  true  ability  values  on 
the  two  dimensions  had  about  equal  correlations  with  the  dimension  1  estimated 
abilities  (0.592  for  dimension  1,  0.597  for  dimension  2).  The  correlations  of 
the  true  abilities  with  the  dimension  2  ability  estimates  were  also  almost 
equal  (0.468  for  dimension  1,  0.507  for  dimension  2)  and  both  were  lower  than 
the  correlations  with  the  dimension  1  estimates.  The  inter-dimension  ability 
correlation  was  0.687  for  the  true  values  and  -0.203  for  the  estimated  values. 


Table  21 

Intercorrelation  Matrix  for  True  and  Estimated 
Ability  Parameters  for  Dataset  5 


1.000 


0.687 

1.000 


Estimated  6, 


0.592 

0.597 

1.000 


The  correlation  between  the  proportion-correct  index  and  the  d-parameter 
was  0.994  (0.992  with  the  d-parameter  estimates).  The  correlation  between  the 
point  blserlal  index  and  the  true  a-parameters  was  0.186  for  dimension  1  and 
-0.142  for  dimension  2.  Wlien  the  a-parameter  estimates  were  used,  these 
correlations  were  0.474  and  -0.252. 

Table  22  shows  the  intercorrelation  matrix  for  the  true  and  estimated 
item  parameters  and  the  rotated  factor  loadings  for  dataset  5.  Interestingly, 
the  factor  loadings  (both  rotations)  are  strongly  related  to  the  estimated  a- 
parameters,  but  not  to  the  true  a-parameters.  The  first  four  eigenvalues  from 
the  principal  components  analysis  of  dataset  5  were  12.32,  1.23,  1.21,  and 
1.18.  The  first  factor  for  this  dataset  is  larger  than  for  the  corresponding 
dataset  from  the  test  1  analyses  (dataset  1). 


Variable 


Item  Parameters 


Estimated 


Factor  Loadings 


Orthogonal  Oblique 

I  II  I  II 


True 

d  1.000 

-0.048 

0. 

.076 

0. 

.997 

0. 

.029 

0. 

.088 

0. 

.113 

-0.061 

0.105 

-0.079 

®1 

1.000 

-0, 

.985 

-0, 

.036 

0. 

.198 

-0. 

.176 

0. 

.244 

-0.195 

0.238 

-0.215 

®2 

1, 

.000 

0. 

.067 

-0. 

.159 

0. 

.167 

-0. 

.215 

0.181 

-0.211 

0.196 

Estimated 

4 

1, 

.000 

0, 

.012 

0, 

.096 

0, 

.116 

-0.067 

0.108 

-0.085 

^1 

1. 

.000 

-0, 

.837 

0, 

.860 

-0.760 

0.851 

-0.806 

®2 

1, 

.000 

-0, 

.869 

0.905 

-0.885 

0.908 

Orthogonal 

I 

1. 

.000 

-0.925 

0.998 

-0.966 

II 

1.000 

-0.949 

0.992 

Oblique 

I 

1.000 

-0.982 

The  point  blserlal  index  had  correlations  of  0.509  (dimension  1)  and 
-0.186  (dimension  2)  with  the  varlmax  rotated  factor  loadings »  and  0.456 
(dimension  1)  and  -0.298  (dimension  2)  with  the  oblique  rotated  factor 
loadings.  The  proportion-correct  index  had  correlations  of  0.085  and  -0.042 
with  the  varlmax  rotated  loadings,  and  0.078  and  -0.057  with  the  oblique 
rotated  factor  loadings.  The  correlation  between  the  proportion-correct  and 
point  biserial  indexes  was  0.120. 

Dataset  6.  Table  23  shows  the  item  parameter  estimates,  item  statistics, 
and  rotated  factor  loadings  for  dataset  6.  These  data  were  generated  using 
test  2  and  a  group  of  examinees  with  an  inter-dimension  ability  correlation  of 
0.50.  The  mean  score  on  test  2  for  this  group  was  24.01,  and  the  standard 
deviation  was  8.54.  The  KR-20  reliability  was  0.87.  The  correlation  between 
factors,  obtained  from  the  oblique  rotation,  was  -0.19, 


Table  23 

Item  Parameter  Estimates,  Item  Statistics,  and  Factor 
Loadings  for  Dataset  6 


Item  Parameter 

Item 

Factor 

Loadings 

Estimates 

Statistics 

Orthogonal 

Oblique 

1 

-1.39 

0 

2 

2.22 

0 

3 

-0.12 

0 

4 

2.78 

0 

5 

-0.86 

0 

6 

-1.36 

0 

7 

0.44 

0 

8 

-0.44 

0 

9 

-0.49 

0 

10 

-1.76 

0 

11 

-0.58 

0 

12 

0.29 

0 

13 

-0.21 

0 

14 

-2.74 

2 

15 

0.40 

0 

16 

1.09 

0 

17 

-0.58 

1 

18 

0.68 

0 

19 

-0.38 

0 

20 

0.65 

0 

21 

-3.04 

1 

22 

-0.24 

0 

23 

-0.69 

0 

24 

-0.21 

1 

25 

-0.73 

0 

26 

1.19 

0 

27 

0.05 

1 

28 

1.54 

1 

29 

0.33 

0 

30 

-1.02 

0 

31 

0.92 

0 

32 

0.28 

0 

33 

-1.02 

0 

34 

0.49 

1 

35 

1.45 

0 

36 

1.06 

1 

37 

-1.21 

1 

38 

0.29 

0 

39 

0.13 

0 

Dll] 


p 

pbls 

.23 

0.26 

.87 

0.26 

.48 

0.34 

.84 

0.27 

.33 

0.31 

.24 

0.29 

.59 

0.36 

.40 

0.30 

.40 

0.32 

.18 

0.26 

.38 

0.36 

.56 

0.38 

.46 

0.33 

.17 

0.28 

.59 

0.33 

.71 

0.32 

.38 

0.36 

.58 

0.35 

.42 

0.33 

.64 

0.32 

.07 

0.21 

.45 

0.36 

.36 

0.33 

.46 

0.36 

.35 

0.31 

.73 

0.31 

.51 

0.35 

.78 

0.29 

.57 

0.36 

.30 

0.33 

.69 

0.30 

.56 

0.34 

.29 

0.30 

.60 

0.35 

.78 

0.29 

.70 

0.32 

.28 

0.34 

.56 

0.34 

.53 

0.36 

-0.05 
-0.13 
-0.18 
-0.42 
0.10 
0.07 
11 
04 

15 
04 
06 

16 
01 
42 
12 
22 
05 
45 
01 
00 
02 
02 
02 
17 
05 
02 
17 
10 
12 
13 
00 
05 
08 
17 
07 
15 
22 
10 
13 


> 


Table  24  shows  the  Intercorrelation  matrix  for  the  true  and  estimated 
item  parameters  for  dataset  6.  The  correlation  between  the  true  and  estimated 
d-parameters  was  0.986,  which  is  somewhat  lower  than  was  the  case  with  dataset 
5.  The  true  a-parameters  for  dimension  1  had  a  correlation  of  0.109  with  the 
dimension  2  estimated  a-values,  while  the  dimension  2  true  a-values  had  a 
correlation  of  0.056  with  the  dimension  1  estimated  a-values.  Again,  there  is 
no  significant  correlation  between  the  true  and  estimated  a-parameters. 


Table  24 

Intercorrelation  Matrix  for  True  and  Estimated 
Item  Parameters  for  Dataset  6 


d 

l.OOO 

-0.048 

0.076 

0.989 

-0.379 

1.000 

-0.985 

-0.021 

-0.065 

®2 

1.000 

0.054 

0.056 

d 

1.000 

-0.469 

1.000 

a2 

Table  25  shows  the  intercorrelation  matrix  for  the  true  and  estimated 
abilities  for  dataset  6.  As  can  be  seen,  the  true  abilities  had  slightly 
higher  correlations  with  the  dimension  2  ability  estimates  than  with  the 
dimension  1  ability  estimates.  Each  true  ability  parameter  had  about  equal 
correlations  with  the  two  sets  of  ability  parameter  estimates.  The  inter- 
dimension  ability  correlations  was  0.493  for  the  true  abilities  and  -0.378  for 
the  estimated  abilities. 


Table  25 

Intercorrelation  Matrix  for  True  and  Estimated 
Ability  Parameters  for  Dataset  6 


Estimated  6^ 


1.000 


0.493 

l.OOO 


0.424 

0.446 

1.000 


There  was  a  correlation  of  0,993  between  the  proportion-correct 
difficulty  index  and  the  true  d-parameter.  The  correlation  was  0.983  for  the 
estimated  d-parameter.  The  point  blserlal  discrimination  index  had 
correlations  of  0.158  and  -0.096  with  the  true  a-parameters  and  correlations 
of  0.032  and  0.137  with  the  estimated  a-parameters. 

Table  26  shows  the  intercorrelation  matrix  for  the  true  and  estimated 
item  parameters  and  the  rotated  factor  loadings  for  dataset  6.  As  was  the 
case  with  dataset  5,  there  was  a  strong  relationship  between  the  estimated  a- 
parameters  and  both  sets  of  rotated  factor  loadings,  but  no  significant 
relationship  between  the  true  a-parameters  and  the  factor  loadings.  The  first 
four  eigenvalues  from  the  principal  components  analysis  of  dataset  6  were 
11.18,  1.28,  1.25,  and  1.22.  There  is  a  large  first  factor,  and  the  second 
factor  is  almost  nonexistent. 
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Item  Parameters 


Factor  Loadings 


Variable 


Estimated 


Orthogonal  Oblique 


d 

^2  ^ 

®2 

I 

II  I 

II 

True 

d  1 

.000 

-0.048 

0.076  0.989  -0.379 

0.335 

-0.370 

0.382  -0.306 

-0.382 

1.000 

-0.985  -0.021  -0.065 

0.109 

-0.008 

0.045  0.036 

-0.030 

1.000  0.054  0.056 

-0.076 

-0.003 

-0.012  -0.021 

0.005 

Estimated 

d^ 

1.000  -0.469 

0.420 

-0.439 

0.449  -0.366 

-0.450 

1.000 

-0.840 

0.953 

-0.886  0.897 

0.928 

1.000 

-0.852 

0.918  -0.659 

-0.902 

Orthogonal 

I 

1.000 

-0.944  0.926 

0.982 

II 

1.000  -0.750 

-0.990 

Oblique 

I 

1.000 

0.837 

The  correlations  of  the  point  biserlals  with  the  varlmax  rotated  loadings 
were  0.074  and  0.194.  With  the  oblique  rotated  loadings,  the  correlations 
were  0.371  and  -0.079.  The  proportion-correct  difficulty  Index  had 
correlations  of  -0.367  and  0.379  with  the  varlmax  rotated  loadings,  and 
correlations  of  -0.303  and  -0.379  with  the  oblique  rotated  loadings.  The 
correlation  of  the  proportion-correct  difficulty  and  point  biserial 
discrimination  Indexes  was  0.094. 


Dataset  7.  Table  27  shows  the  Item  parameter  estimates,  item  statistics, 
and  rotated  factor  loadings  for  dataset  7 .  These  data  were  generated  using 
test  2  and  a  group  of  examinees  having  an  inter-dimension  ability  correlation 
of  0.35.  The  mean  score  on  test  2  for  this  group  was  24.07,  and  the  standard 
deviation  was  8.20.  The  KR-20  reliability  was  0.86.  The  correlation  between 
factors,  obtained  from  the  oblique  rotation,  was  0.23. 
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Table  27 

Item  Parameter  Estimates,  Item  Statistics,  and  Factor 
Loadings  for  Dataset  7 


Item  Parameter 


Estimates 


Factor  Loadings 


Statistics  Orthogonal 


.22 

0.25 

0.18 

.88 

0.24 

0.25 

.47 

0.34 

0.36 

.84 

0.27 

0.11 

.32 

0.30 

0.34 

.24 

0.27 

0.34 

.59 

0.35 

0.34 

.41 

0.28 

0.32 

.40 

0.29 

0.27 

.18 

0.25 

0.29 

.39 

0.34 

0.38 

.56 

0.36 

0.28 

.46 

0.31 

0.37 

.17 

0.26 

0.57 

.59 

0.32 

0.30 

.72 

0.30 

0.16 

.38 

0.34 

0.37 

.59 

0.34 

0.10 

.42 

0.31 

0.28 

.64 

0.31 

0.32 

.07 

0.20 

0.17 

.45 

0.34 

0.38 

.36 

0.32 

0.33 

.46 

0.35 

0.38 

.35 

0.29 

0.38 

.74 

0.29 

0.45 

.52 

0.34 

0.54 

.79 

0.27 

0.36 

.57 

0.34 

0.26 

.30 

0.30 

0.28 

.70 

0.29 

0.36 

.56 

0.32 

0.46 

.29 

0.28 

0.34 

.59 

0.33 

0.48 

.78 

0.26 

0.32 

.71 

0.30 

0.45 

.28 

0.33 

0.55 

.56 

0.32 

0.28 

.53 

0.34 

0.28 

.38 

0.31 

0.33 

.82 

0.26 

0.02 

.44 

0.33 

0.32 

Oblique 


V  W  V  >  ^ 


Item  Parameter 


Item 


Factor  Loadings 


Estimates 


Statistics  Orthogonal 


Oblique 


43 

-0.82 

0.54 

1.10 

0.34 

0.32 

0.16 

0.49 

0.36 

44 

0.34 

0.85 

0.96 

0.57 

0.36 

0.36 

0.36 

0.48 

45 

0.97 

0.96 

0.73 

0.69 

0.31 

0.41 

0.25 

0.49 

46 

0.81 

0.56 

0.86 

0.67 

0.29 

0.21 

0.39 

0.36 

47 

-1.81 

0.80 

0.95 

0.18 

0.29 

0.35 

0.32 

0.46 

48 

-1.98 

1.39 

0.51 

0.18 

0.28 

0.56 

0.09 

0.56 

49 

-0.70 

0.74 

1.13 

0.36 

0.36 

0.30 

0.42 

0.46 

'50 

-0.78 

0.88 

0.86 

0.34 

0.34 

0.40 

0.30 

0.50 

Table  28  contains  the  Intercorrelatlon  matrix  for  the  true  and  estimated 
Item  parameters  for  dataset  7.  The  correlation  between  the  true  and  estimated 
d-parameters  was  0.988,  which  Is  about  the  same  as  was  obtained  for  dataset 
6.  The  true  a-parameters  for  dimension  1  had  a  correlation  of  0.049  with  the 
dimension  2  estimated  a-values,  and  the  dimension  2  true  a-parameters  had  a 
correlation  of  0.061  with  the  dimension  I  estimated  a-values. 


Table  28 


Intercorrelatlon  Matrix  for  True  and  Estimated 
Item  Parameters  for  Dataset  7 


Table  29  contains  the  Intercorrelatlon  matrix  for  the  true  and  estimated 
ability  parameters  for  dataset  7.  The  values  In  this  table  follow  a  pattern 
much  like  what  was  found  for  dataset  6.  Each  true  ability  had  about  equal 
correlations  with  the  two  sets  of  estimates,  and  both  had  slightly  higher 
correlations  with  the  dimension  2  estimates  than  with  the  dimension  1 


estimates.  The  inter-dimension  ability  correlation  was  0.334  for  the  true 
abilities  and  -0.380  for  the  estimates. 


Table  29 


Intercorrelation  Matrix  for  True  and  Estimated 
Ability  Parameters  for  Dataset  7 


Variable 

True 

Estimated 

01  02 

®1 

®2 

True 

1.000  0.334 

0.381 

0.430 

1.000 

0.411 

0.450 

Estimated 

®2 

1.000 

-0.380 

1.000 

There  was  a  correlation  of  0.993  between  the  proportion-correct 
difficulty  index  and  the  true  d-parameter.  The  correlation  was  0.983  for  the 
estimated  d-parameter.  The  point  biserlal  discrimination  index  had 
correlations  of  0.170  and  -0.107  with  the  true  a-parameters  and  correlations 
of  0.010  and  0.051  with  the  estimated  a-parameters. 

Table  30  shows  the  Intercorrelation  matrix  for  the  true  and  estimated 
item  parameters  and  the  rotated  factor  loadings  for  dataset  7.  As  was  the 
case  with  datasets  5  and  6,  there  was  a  strong  relationship  between  the 
estimated  a-parameters  and  both  sets  of  rotated  factor  loadings »  but  no 
significant  relationship  between  the  true  a-parameters  and  the  factor 
loadings.  The  first  four  eigenvalues  from  the  principal  components  analysis 
of  dataset  7  were  10.33,  1.31,  1.26,  and  1.23.  There  is  a  large  first  factor, 
and  the  second  factor  is  almost  nonexistent. 


Item  Parameters 

Factor  Loadings 

Variable 

True  Estimated 

Orthogonal  Oblique 

True 

d  1.000 

-0.048  0.076 

0.987 

-0.323 

0.364 

-0.228 

0.248 

-0.192 

0.243 

®1 

1.000  -0.985 

-0.043 

-0.069 

0.049 

0.006 

0.040 

0.043 

0.021 

^2 

1.000 

0.075 

0.061 

-0.018 

0.003 

-0.027 

-0.018 

-0.017 

Estimated 

d 

1.000 

-0.433 

0.468 

-0.322 

0.339 

-0.279 

0.337 

®1 

1.000 

-0.837 

0.924 

-0.872 

0.885 

-0.908 

4 

1.000 

-0.832 

0.910 

-0.695 

0.890 

Orthogonal 

I 

1.000 

-0.940 

0.961 

-0.980 

II 

1.000 

-0.808 

0.989 

Oblique 

I 

1.000 

-0.887 

The  correlations  of  the  point  biserials  with  the  varimax  rotated  loadings 
were  0.2A6  with  0.030.  With  the  oblique  rotated  loadings,  the  correlations 
were  0.448  and  -0.089.  The  proportion-correct  difficulty  index  had 
correlations  of  -0.241  and  0.262  with  the  varimax  rotated  loadings,  and 
correlations  of  -0.204  and  0.257  with  the  oblique  rotated  loadings.  The 
correlation  of  the  proportion-correct  difficulty  and  point  biserial 
discrimination  indexes  was  0.097. 


Dataset  8.  Table  31  shows  the  item  parameter  estimates,  item  statistics, 
and  rotated  factor  loadings  for  dataset  8.  These  data  were  generated  using 
test  2  and  a  group  having  an  inter-dimension  ability  correlation  of  0.00.  The 
mean  score  on  test  2  for  this  group  was  24.18,  and  the  standard  deviation  was 
7.32.  The  KR-20  reliability  was  0.82.  The  correlation  between  factors, 
obtained  from  the  oblique  rotation,  was  -0.57. 


b 


u 


Item  Parameter 


Estimates 


Factor  Loadings 


Statistics  Orthogonal 


Oblique 


26 

1.24 

0.96 

0.56 

0.75 

0.24 

27 

0.10 

1.18 

0.48 

0.52 

0.28 

28 

1.54 

0.67 

0.72 

0.80 

0.22 

29 

0.34 

0.76 

0.95 

0.57 

0.31 

30 

-1.02 

0.71 

0.81 

0.29 

0.27 

31 

0.98 

0.70 

0.74 

0.71 

0.25 

32 

0.30 

0.97 

0.54 

0.56 

0.27 

33 

-0.96 

0.73 

0.54 

0.30 

0.24 

34 

0.49 

1.06 

0.66 

0.60 

0.30 

35 

1.47 

0.86 

0.56 

0.79 

0.23 

36 

1.00 

0.70 

0.73 

0.71 

0.25 

37 

-1.68 

2.00 

0.17 

0.27 

0.28 

38 

0.33 

0.53 

0.99 

0.57 

0.28 

39 

0.17 

0.63 

0.95 

0.54 

0.30 

40 

-0.56 

0.68 

0.71 

0.38 

0.26 

41 

2.66 

0.0 

2.00 

0.82 

0.22 

42 

-0.28 

0.77 

0.63 

0.44 

0.27 

43 

-0.81 

0.68 

0.88 

0.33 

0.28 

40 

0.14 

0.43 

0.02 

48 

0.08 

0.54 

0.11 

23 

0.31 

0.17 

-0.25 

25 

0.41 

0.16 

-0.37 

29 

0.30 

0.25 

-0.22 

31 

0.23 

0.30 

-0.13 

43 

0.12 

0.47 

0.05 

36 

0.15 

0.38 

-0.02 

43 

0.19 

0.45 

-0.03 

34 

0.18 

0.34 

-0.06 

33 

0.23 

0.32 

-0.12 

54 

0.06 

0.62 

0.16 

21 

0.37 

0.13 

-0.34 

20 

0.42 

0.11 

-0.40 

28 

0.28 

0.25 

-0.20 

06 

0.60 

-0.25 

-0.71 

31 

0.25 

0.29 

-0.16 

20 

0.41 

0.11 

-0.38 

35 

0.29 

0.32 

-0.19 

37 

0.19 

0.37 

-0.06 

06 

0.47 

-0.07 

-0.51 

47 

0.28 

0.29 

-0.19 

47 

0.06 

0.54 

0.13 

27 

0.41 

0.20 

-0.35 

42 

0.19 

0.44 

-0.03 

Table  32  contains  the  Intercorrelatlon  matrix  for  the  true  and  estimated 
Item  parameters  for  dataset  8.  The  correlation  between  the  true  and  estimated 
d-parameters  was  0.989.  The  true  and  estimated  a-values  for  dimension  1  had  a 
correlation  of  0.11,  while  for  dimension  2  the  correlation  was  0.037. 


1.000 


“1 

^2 

Estimated  d 


-0.048 

1.000 


0.076 

-0.985 

1.000 


0.989 

-0.074 

0.104 

1.000 


-0.331 

0.011 

-0.017 

-0.428 

1.000 


0.370 

-0.005 

0.037 

0.464 

-0.784 

1.000 


Table  33  shows  the  Intercorrelation  matrix  for  the  true  and  estimated 
ability  parameters  for  dataset  8.  The  same  pattern  is  present  as  was  found 
for  the  other  test  2  datasets.  Both  sets  of  true  abilities  had  about  equal 
correlations  with  the  two  sets  of  estimates.  The  inter-dimension  ability 
correlation  was  -0.036  for  the  true  values  and  -0.466  for  the  estimates. 


Table  33 

Intercorrelation  Matrix  for  True  and  Estimated 
Ability  Parameters  for  Dataset  8 


1.000 

-0.036 

0.288 

0.309 

1.000 

0.343 

0.344 

e? 

1.000 

-0.466 

«2 

1.000 

The  correlation  between  the  true  d-parameter  and  the  proportion-correct 
index  was  0.993.  The  correlation  between  the  proportion-correct  index  and  the 
estimated  d-values  was  0.986.  The  correlations  of  the  point  biserlal  index 
and  the  a-values  were  0.140  and  -0.083  for  the  true  values,  and  0.203  and 
0.084  for  the  estimates. 

Table  34  shows  the  Intercorrelation  matrix  for  the  true  and  estimated 
item  parameters  and  the  two  sets  of  rotated  factor  loadings  obtained  for 
dataset  8.  As  has  been  the  pattern  with  the  test  2  datasets,  there  is  a 
strong  relationship  between  the  estimated  a-values  and  both  sets  of  loadings, 
but  no  correlation  between  the  true  a-values  and  the  factor  loadings.  The 
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first  four  eigenvalues  from  the  principal  components  analysis  of  dataset  8 
were  8.19,  1.39,  1.30,  and  1.29. 


Table  34 


Intercorrelation  Matrix  for  True  and  Estimated  Item 
Parameters  and  Factor  Loadings  for  Dataset  8 


Variable 

Item  Parameters 

Factor  Loading 

s 

True 

Estimated 

Orthogonal  Oblique 

d 

ai  ^2 

d  aj 

32 

I 

II  I 

II 

True 

d  1 

.000 

-0.048  0.076 

0.989  -0.331 

0.370 

-0.308 

0.308  -0.312 

-0.312 

1.000  -0.985 

-0.074  0.011 

-0.005 

-0.035 

0.072  -0.043 

-0.064 

ap 

1.000 

0.104  -0.017 

0.037 

0.037 

-0.056  0.042 

0.052 

Estimated 

d 

1.000  -0.428 

0.464 

-0.390 

0.380  -0.392 

-0.388 

1.000 

-0.784 

0.887 

-0.786  0.876 

0.823 

a2 

1.000 

-0.818 

0.888  -0.844 

-0.882 

Orthogonal 

I 

1.000 

-0.929  0.997 

0.960 

II 

1.000  -0.956 

-0.995 

Oblique 

I 

1.000 

0.980 

II 

1.000 

The  correlations  of  the  point  blserial  Index  with  the  rotated  factor 
loadings  were  0.243  and  0.074  for  the  varlmax  rotation,  and  0.175  and  0.006 
for  the  oblique  rotation.  For  the  proportion-correct  difficulty  index  the 
correlations  were  -0.324  and  0.330  with  the  varlmax  rotated  loadings,  and 
-0.330  and  -0.333  with  the  oblique  rotated  factor  loadings.  The  correlation 
between  the  point  biserlals  and  proportion-correct  values  was  0.073. 

Discussion 


The  purpose  of  this  study  was  to  examine  the  effects  of  correlated 
abilities  on  observed  test  characteristics,  and  to  explore  the  implications  of 
correlated  abilities  for  multidimensional  item  response  theory,  or  MIRT, 
analysis  when  a  model  is  used  that  does  not  explicitly  account  for  such  a 
correlation.  The  approach  taken  was  to  generate  simulation  data  with  known 
true  parameters,  using  varying  levels  of  correlation  between  abilities,  and  to 
analyze  the  data  using  a  number  of  different  test  analysis  procedures.  The 
procedures  selected  were  item  analysis,  principal  component  analysis,  and  MIRT 
analysis.  In  addition,  correlational  analyses  were  performed  to  explore  the 
relationship  of  obtained  statistics  and  parameter  estimates  to  the  true 
parameters,  as  well  as  the  Interrelationships  among  the  item  parameter 
estimates  and  traditional  item  statistics.  All  of  these  analyses  were 
performed  for  two  different  tests.  One  test  was  comprised  of  two  relatively 
Independent  dimensions  (each  item  discriminating  on  only  one  of  the 
dimensions),  while  the  other  test  was  comprised  of  two  correlated  dimensions 
(each  item  discriminating  at  least  moderately  on  both  dimensions).  The 
analyses  of  these  two  tests  will  be  discussed  separately,  and  then  an  attempt 
will  be  made  to  Integrate  the  results  of  the  two  sets  of  analyses. 
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Test  1  Analyses 

In  test  1,  an  attempt  was  made  to  use  two  relatively  unidimensional 
subsets  of  items.  One  subset  of  items  discriminated  fairly  highly  on  the 
first  dimension,  and  very  poorly  on  the  second.  The  second  subset  of  items 
discriminated  fairly  highly  on  the  second  dimension  and  relatively  poorly  on 
the  first.  Thus,  the  test  had  two  relatively  independent  factors.  In  an 
attempt  to  evaluate  the  effects  of  correlated  abilities  for  such  a  test,  the 
three  types  of  test  analysis  procedures  were  applied,  and  the  results 
analyzed.  The  results  for  each  type  of  test  analysis  procedure  will  now  be 
discussed  separately. 

Item  Analysis  Results.  The  one  clear  pattern  which  emerged  from  the  item 
analyses  performed  on  these  data  was  the  decline  of  the  test  KR-20  reliability 
with  the  decline  of  the  correlation  between  ability  dimensions.  This  trend  is 
summarized  in  Table  35,  which  shows  a  drop  in  reliability  from  0.86  to  0.77 
when  the  inter-dimension  ability  correlation  dropped  from  0.70  to  0.00.  This 
is  an  indication  that  an  Increased  correlation  between  ability  dimensions 
results  in  more  common  variance,  which  in  turn  yields  a  higher  KR-20.  The 
fact  that  the  items  were  constructed  to  have  a  relatively  low  inter- 
dimensional  correlation  may  have  somewhat  mitigated  this  effect,  but  it  did 
not  eliminate  it. 


Table  35 


Relationship  Between  Ability  Correlation 
and  Test  Reliability  for  Test  1 


Dataset 

Ability  Correlation 

KR-20 

1 

0.70 

0.86 

2 

0.50 

0.84 

3 

0.35 

0.82 

4 

0.00 

0.77 

Principal  Component  Analysis  Results.  One  pattern  evident  in  the  factor 
analysis  results  was  the  decline  in  factor  correlation  with  the  decline  in 
ability  correlation.  This  pattern  was  similar  to  that  found  for  the  KR-20 
analyses,  and  is  also  indicative  of  the  increased  multidimensionality  of  the 
test  data  (decreased  size  of  the  common  component).  The  pattern  Is 
illustrated  in  Table  36. 


A  similar  sort  of  pattern  was  evident  In  the  eigenvalues  resulting  from 
the  principal  components  analyses.  The  first  four  eigenvalues  for  each  set  of 
response  data  are  shown  In  Table  37.  As  can  be  seen,  as  the  ability 
correlation  decreased,  so  did  the  size  of  the  first  eigenvalue.  At  the  same 
time,  the  size  of  the  second  eigenvalue  Increased. 

Table  37 


Relationship  Between  Ability  Correlation 
and  Eigenvalues  for  Test  1 


Dataset 

Ability 

Eigenvalues 

Correlation 

E2 

E3 

E4 

1 

0.70 

10.01 

1.50 

1.31 

1.27 

2 

0.50 

9.09 

1.79 

1.30 

1.28 

3 

0.35 

7.92 

2.08 

1.43 

1.29 

4 

0.00 

6.32 

2.74 

1.42 

1.32 

Another  trend  found  in  the  results  of  these  analyses  was  the  tendency 
toward  an  increase  in  the  correlations  between  the  true  item  discriminations 
and  both  the  orthogonal  and  oblique  rotated  factor  loadings  as  the  ability 
correlation  decreased.  This  tendency  is  shown  in  Table  38.  Bear  in  mind  that 
the  correlations  shown  are  not  always  matched  on  dimensions.  That  is,  in  some 
cases  the  dimension  1  a-value  correlation  with  the  factor  II  loadings  is 
shown. 


Ability 


Orthogonal 


Oblique 


Dataset 


elation 

I 

II 

I 

.70 

0.86 

0.86 

0.87 

.50 

0.94 

0.90 

0.93 

.35 

0.96 

0.95 

0.96 

.00 

0.96 

0.95 

0.96 

MIRT  Analysis  Results.  As  the  correlation  between  ability  dimensions 
decreased,  there  was  a  slight  increase  in  the  correlations  between  the  true 
and  estimated  item  parameters.  That  is  to  say,  the  estimation  program  was 
better  able  to  recover  the  true  parameters  when  the  ability  dimensions  were 
leas  correlated.  This  trend  is  shown  in  Table  39.  Note  that  the  dimensions 
did  not  always  match.  That  is,  some  of  the  correlations  reported  in  Table  39 
for  the  discrimination  values  are  actually  correlations  between  the  true  a- 
values  on  one  dimension  and  the  estimated  a-values  for  the  other  dimension. 


Table  39 

Relationship  Between  Ability  Correlation 
and  the  Correlation  Between  True  and  Estimated 
Item  Parameters  for  Test  1 


Ability 


Item  Parameter 


Dataset 

Correlation 

d 

1 

0.70 

0.996 

0.73 

2 

0.50 

0.998 

0.87 

3 

0.35 

0.999 

0.92 

4 

0.00 

0.999 

0.92 

Another  interesting  result  from  the  MIRT  analyses  Involved  the 
correlation  between  the  discrimination  parameters.  For  test  I  the  correlation 
between  the  true  dimension  1  and  dimension  2  a-values  was  -0.987.  Table  40 
shows  the  a-value  correlations  for  the  estimated  a-values  for  the  four 
datasets  for  test  1.  Note  that  the  correlation  is  well  below  (in  absolute 
value)  the  correlation  for  the  true  values  for  dataset  1,  but  as  the  ability 
correlation  decreases  the  obtained  a-value  correlation  more  nearly  approaches 
the  true  value. 


Dataset 


Ability  Correlation 


a-Value  Correlation 


1 

0.70 

-0.841 

2 

0.50 

-0.865 

3 

0.35 

-0.936 

4 

0.00 

-0.937 

Table  41  summarizes  the  correlations  between  the  true  and  estimated 
ability  parameters  for  test  1.  As  can  be  seen,  the  correlations  Increase  as 
the  inter-dimension  ability  correlation  decreases.  The  estimation  program  was 
better  able  to  recover  the  true  examinee  abilities  when  the  ability  dimensions 
were  less  correlated.  This  is  consistent  with  the  results  obtained  for  the 
item  parameters. 

Table  41 


Relationship  Between  Ability  Correlation 
and  the  Correlations  Between  the  True  and  Estimated 
Ability  Parameters  for  Test  1 


Dataset 

Ability 

Ability 

Parameter 

Correlation 

®2 

1 

0.70 

0.70 

0.67 

2 

0.50 

0.74 

0.72 

3 

0.35 

0.78 

0.77 

4 

0.00 

0.81 

Table  42  shows  the  correlations  between  the  two  ability  dimensions  for 
the  ability  parameter  estimates  for  the  four  datasets  for  test  1.  Also  shown 
are  the  actual  true  ability  correlations  obtained  for  the  four  datasets  (as 
opposed  to  the  true  correlations  for  the  populations  from  which  examinees  were 
selected).  As  can  be  seen,  the  sample  correlations  for  the  true  abilities 
were  quite  close  to  the  true  population  values.  In  every  case,  however,  the 
correlation  for  the  ability  parameter  estimates  is  very  near  0.0.  Regardless 
of  the  true  correlation  between  ability  dimensions,  the  ability  parameter 
estimate  dimensions  are  forced  to  be  uncorrelated. 
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Table  42 

Relationship  Between  Ability  Correlation 
and  Inter-Dimension  Ability  Estimate 
Correlation  for  Test  1 


Dataset 

Ability  Correlation 

(Population) 

Ability  Correlation 

(Sample) 

Ability  Estimate 

Correlation 

1 

0.70 

0.68 

-0.14 

2 

0.50 

0.49 

-0.15 

3 

0.35 

0.35 

-0.09 

4 

0.00 

0.01 

-0.05 

Test  2  Analyses 

In  test  2  items  were  selected  to  have  at  least  moderately  high 
discriminations  on  both  dimensions.  Thus,  the  test  was  constructed  to  have 
somewhat  correlated  dimensions.  The  same  analyses  performed  on  test  1  were 
then  run  on  test  2.  Again,  the  results  for  each  type  of  test  analysis 
procedure  will  be  discussed  separately. 

Item  Analysis  Results.  The  only  pattern  discernible  among  the  item 
analysis  results  for  the  test  2  datasets  was  a  slight  decline  in  KR-20 
reliability  with  a  decrease  in  the  correlation  between  ability  dimensions. 
This  trend  can  be  seen  in  the  data  shown  in  Table  43.  The  trend  is  less 
dramatic  than  was  the  case  with  the  test  1  data,  however.  In  the  test  1 
datasets  the  principal  component  was  due  primarily  to  the  ability  dimension 
correlation,  while  in  the  test  2  data  the  principal  component  was  at  least 
partially  due  to  the  nature  of  the  items.  Thus,  a  decline  in  ability 
correlation  did  not  have  as  great  an  impact  on  its  size. 


Table  43 

Relationship  Between  Ability  Correlation 
and  Test  Reliability  for  Test  2 


Dataset 

Ability  Correlation 

KR-20 

5 

0.70 

0.89 

6 

0.50 

0.87 

7 

0.35 

0.86 

8 

0.00 

0.82 

Principal  Component  Analysis  Results.  The  pattern  of  factor  correlations 
was  rather  confusing  for  the  test  2  datasets.  These  values  are  shown  In  Table 
44. 


Dataset 


Ability  Correlation 


Factor  Correlation 


As  can  be  seen,  there  is  no  systematic  relationship  between  ability 
correlation  and  factor  correlation.  To  be  consistent  with  the  previous 
results,  the  factor  correlation  should  have  declined  slightly  with  the 
decrease  in  ability  correlation.  The  results  for  datasets  5  and  8  are 
consistent  with  this,  but  the  results  for  datasets  6  and  7  are  quite 
Inconsistent  with  this.  As  yet,  no  satisfactory  explanation  for  this 
phenomenon  has  been  determined. 

Table  45  shows  the  trend  in  eigenvalues  as  ability  correlation  decreased 
for  the  test  2  data.  As  can  be  seen,  these  results  are  much  more  consistent 
with  the  item  analysis  results.  There  was  a  slight  decrease  in  the  size  of 
the  first  eigenvalue  with  the  decrease  in  ability  correlation.  The  decrease 
is  more  marked  from  dataset  7  to  8  than  between  the  other  tests,  but  so  was 
the  decline  in  KR-20.  There  was  a  negligible  increase  in  the  size  of  the 
second  eigenvalue  as  the  ability  correlation  decreased. 


Table  45 

Relationship  Between  Ability  Correlation 
and  Eigenvalues  for  Test  2 


Dataset 


Ability 


Correlation 


Eigenvalues 


.70 

12.32 

1.23 

1.21 

.50 

11.18 

1.28 

1.25 

.35 

10.33 

1.31 

1.26 

.00 

8.19 

1.39 

1.30 

A  pattern  found  for  the  test  2  analyses  which  is  in  marked  contrast  to 
the  results  for  test  1  Involves  the  correlations  between  the  true  a-values  and 
the  factor  loadings.  For  test  1  there  was  a  tendency  for  an  Increase  in 
correlations  between  a-values  and  factor  loadings  as  the  ability  correlation 
decreased.  All  of  the  a-value-f actor  loading  correlations  were  relatively 


high,  though.  For  the  test  2  data,  the  true  a-value-f actor  loading 
correlations  were  all  around  0.0,  regardless  of  the  ability  correlation. 


MIRT  Analysis  Results.  As  the  correlation  between  ability  dimensions 
decreased,  the  correlation  between  the  true  and  estimated  d-parameters 
decreased  slightly,  except  for  dataset  8,  for  which  the  correlation  was  the 
same  as  for  dataset  6.  The  correlation  between  the  true  and  estimated 
a-parameters  was  essentially  0.0  for  all  four  test  2  datasets. 


As  the  correlation  between  ability  dimensions  decreased,  the  correlations 
between  the  two  a-parameter  estimates  for  the  two  dimensions  diverged  from  the 
correlation  between  the  true  a-parameters.  This  trend  is  shown  In  Table  46. 
The  true  a-parameter  correlation  for  test  2  was  -0.985. 


Table  46 


Relationship  Between  Ability  Correlation 
and  Inter-Dimension  a-Value  Correlation 


Dataset 


Ability  Correlation 


a-Value  Correlation 


-0.837 

-0.840 

-0.836 

-0.784 


Table  47  shows  the  relationship  between  true  ability  correlation  and  the 
correlation  between  the  true  and  estimated  ability  parameters.  As  can  be 
seen,  as  the  true  ability  correlation  decreased,  so  did  the  correlations 
between  the  true  and  estimated  ability  parameters. 


Table  47 


Relationship  Between  Ability  Correlation 
and  the  Correlations  Between  the  True  and  Estimated 
Ability  Parameters  for  Test  2 


-■••• 

5 

0 

.70 

0.592 

.>  6 

0 

.50 

0.424 

0 

.35 

0.381 

8 

r-TT— 

0 

.00 

0.288 

Table  48  shows  a  very  interesting  pattern  involving  the  inter-dimension 
ability  estimate  correlations  for  test  2.  As  can  be  seen,  the  sample  true 
ability  dimension  correlations  are  quite  close  to  the  target  population 


valnes.  For  the  ability  estimates,  however,  the  correlation  differs 
substantially  from  the  true  ability  correlation.  In  every  case  the 
correlation  is  nej^ative,  and  it  becomes  more  negative  as  the  true  value 
approached  0.0. 

Table  48 


Relationship  Between  Ability  Correlation 
and  Inter-Dimension  Ability  Estimate 
Correlation  for  Test  2 


Dataset 

Ability  Correlation 

(Population) 

Ability  Correlation 

(Sample) 

Ability  Estimate 

Correlation 

5 

0.70 

0.69 

-0.20 

6 

0.50 

0.49 

-0.38 

7 

0.35 

0.33 

-0.38 

8 

0.00 

-0.04 

-0.47 

Overall  Results 

It  is  quite  clear  that  it  is  not  enough  to  talk  about  the  'dimensionality 
of  a  test',  or  about  whether  the  'dimensions  of  a  test'  are  correlated.  There 
are  two  distinct  concepts  Involved,  and  they  play  quite  different  roles  in 
determining  the  latent  structure  of  response  data.  The  first  concept  is 
latent  item  structure,  and  the  second  is  latent  ability  structure. 

The  latent  structure  of  a  test  item  refers  to  the  number  and 
interrelationships  of  the  dimensions  required  for  performance  on  the  item.  In 
this  research,  two  types  of  latent  item  structure  we^'e  employed.  For  test  1, 
each  item  required  basically  only  one  dimension.  The  first  half  of  the  items 
required  the  first  dimension,  while  the  second  half  required  the  second 
dimension.  Thus,  there  were  two  dimensions  underlying  the  test,  each  of  which 
operated  relatively  independently  of  the  other. 

For  test  2,  each  item  required  two  dimensions.  For  some  items  the  first 
dimension  was  more  dominant  than  the  second,  while  for  the  remaining  items  the 
second  dimension  was  dominant.  Since  all  items  required  both  dimensions,  the 
two  dimensions  did  not  operate  as  Independently  as  for  test  1. 

The  latent  ability  structure  of  an  examinee  refers  to  the  number  and 
interrelationships  of  the  dimensions  underlying  the  examinee's  responses.  In 
this  study  the  latent  ability  space  was  always  two-dimensional,  but  the 
correlation  of  the  two  examinee  latent  ability  dimensions  varied. 

It  is  the  interaction  of  these  two  concepts  which  determines  the  latent 
structure  of  response  data.  When  the  item  dimensions  operate  relatively 
Independently,  the  dimensionality  of  the  response  data  depends  to  a  great 
extent  on  the  latent  ability  structure  of  the  examinees.  Correlated  abilities 
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tend  to  yield  response  data  with  a  single  dominant  component  or,  viewed 
differently,  response  data  with  correlated  latent  dimensions.  Uncorrelated 
abilities  tend  to  yield  response  data  with  relatively  uncorrelated  latent 
dimensions. 


When  the  latent  item  dimensions  do  not  operate  independently,  the  effect 
of  the  latent  ability  structure  is  less  pronounced.  The  effect  of  the 
correlation  between  latent  ability  dimensions  is  the  same,  but  less  extreme. 

The  effect  of  the  interaction  of  latent  ability  and  item  structures  has 
serious  implications  for  the  analysis  of  test  data  and,  perhaps  more 
Importantly,  for  test  development.  Clearly  it  is  not  sufficient  to  consider 
only  item  characteristics  when  constructing  or  analyzing  a  test.  It  is  not 
appropriate  to  assume  that  the  latent  ability  structure  is  determined  by  item 
characteristics.  It  is  necessary  to  consider  how  the  two  interact  to  produce 
a  latent  structure  for  test  data. 

The  results  of  this  stuiy  also  have  important  implications  for  the 
application  of  MIRT  methodology.  The  presence  of  correlated  abilities 
certainly  had  pronounced  effects  on  the  results  obtained  from  the  application 
of  the  MIRT  model  selected  for  this  study.  The  most  important  finding  of  this 
study  regarding  the  use  of  MIRT  methodology  involves  the  inability  of  the  MIRT 
model  estimation  program  to  recover  the  true  dimensions  when  the  dimensions 
were  correlated. 

When  latent  item  dimensions  are  independent,  the  procedure  works  fairly 
well,  even  when  the  latent  ability  dimensions  are  correlated.  However,  an 
Increased  correlation  between  latent  ability  dimensions  does  lower  the 
correlations  between  the  true  and  estimated  abilities.  This  does  not 
necessarily  mean  that  the  estimation  process  breaks  down.  It  simply  means 
that  the  nature  of  the  ability  dimensions  recovered  by  estimation  is  somewhat 
different  than  for  the  true  dimensions.  It  is  entirely  possible  that  the 
estimated  dimensions  are  in  some  sense  a  rotation  of  the  true  dimensions  due 
to  the  fact  that  the  estimation  procedure  and/or  model  does  not  explicitly 
account  for  inter-dimensional  correlations. 

When  latent  item  dimensions  are  not  independent,  the  recovered  dimensions 
are  different  from  the  true  dimensions  regardless  of  whether  or  not  the  latent 
ability  dimensions  are  correlated.  Again,  some  type  of  rotation  of  the  latent 
ability  structure  might  be  involved.  Under  the  circumstances,  if  MIRT 
methodology  is  to  be  viable,  research  on  this  question  must  be  conducted.  If 
a  rotation  is  involved,  it  is  imperative  that  its  nature  be  discovered. 
Moreover,  if  MIRT  parameters  are  to  be  invariant  and  interpretable ,  it  seems 
likely  that  it  will  be  necessary  to  develop  something  analagous  to  factor 
rotations  in  factor  analysis. 

Summary  and  Conclusions 

A  study  was  conducted  to  assess  the  effects  of  correlated  abilities  on 
test  characteristics,  and  to  explore  the  effects  of  correlated  abilities  on 
the  use  of  a  multidimensional  item  response  theory  model  which  does  not 
explicitly  account  for  such  a  correlation.  Two  tests  were  constructed.  One 
test  had  two  relatively  unidimensional  subsets  of  items,  while  the  other  had 
items  that  were  all  two-dimensional.  For  each  test  response  data  were 
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generated  according  to  a  multidimensional  two-parameter  logistic  model  using 
four  groups  of  examinees.  The  groups  of  examinees  differed  in  the  degree  of 
Inter-dlmension  ability  correlation. 

To  evaluate  the  effects  of  correlated  abilities  on  test  characteristics, 
the  simulated  response  data  were  analyzed  using  Item  analysis  and  principal 
component  analysis  techniques.  To  assess  the  effects  of  correlated  abilities 
on  the  use  of  the  multidimensional  model,  the  parameters  of  the  model  were 
estimated,  and  the  estimates  were  compared  to  the  true  parameters. 

The  results  of  this  study  Indicated  that  the  presence  of  correlated 
abilities  has  Important  Implications  for  the  characteristics  of  test  data,  and 
for  the  application  of  multidimensional  Item  response  theory  models.  It  was 
concluded  that  It  Is  necessary  to  consider  latent  Item  structure  as  well  as 
latent  ability  structure  In  test  construction  and  analysis.  It  was  also 
concluded  that  use  of  multidimensional  Item  response  theory  models  that  do  not 
explicitly  account  for  correlated  abilities  may  not  yield  accurate  information 
about  the  nature  of  underlying  dimensions.  It  was  suggested  that  research 
should  be  conducted  to  determine  the  relationship  between  the  observed  and 
true  correlation  between  abilities,  and  to  perhaps  develop  an  Item  response 
theory  analogue  to  factor  rotation. 
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Code  1001 

Of'Ke  of  Na.al  Research 
BOC  N.  Cuincy  St. 

Arlington,  VA  22217 

1  Najor  Frank  Yohannan,  USNC 
Headquarters,  Narine  Corps 
(Code  NPl-20) 

Hasringtc.*,  DC  203BC 


Aray 

1  Dr.  Kent  Eaton 
Aroy  Research  Institute 
5001  Eisenhouer  Blvd. 

Alexandria  ,  VA  22333 

1  Dr.  Nyron  Fischl 

U.S.  Aroy  Research  Institute  for  the 
Social  and  Behavioral  Sciences 
5001  Eisenhouer  Avenue 
Alevandna,  VA  22333 

I  Dr.  Clesser  Nartm 
Aroy  Research  Institute 
5001  Eisenhouer  Blvd. 

Alexandria,  VA  22333 

1  Dr.  Nilliai  E.  Nordtrock 
FHC-ADCO  Box  25 
APO.  NY  0971 C 

1  Dr,  Ha-oli  F.  C’Neil.  Jr. 

Director,  Training  Research  Lab 
Ariy  Research  Institute 
5001  Eisenhouer  Avenue 
Alexandria,  VA  22333 
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]  CMundtr,  O.S.  Ariy  RttHrtk  Inftitute 
for  tht  Miavioril  t  Social  Scitncis 
ATTN:  PERI-IR  (Dr.  Judith  Orasanu) 

9001  EistnhoMT  Dvtnut 
Oliiandria,  VA  22333 

I  Nr.  Robtrt  Ross 

U.S.  Aroy  Rosearch  Institute  for  the 
Social  and  letiavioral  Sciences 
300!  EistnhoMr  Avenue 
Alciandria,  VA  22333 

1  Dr.  Robert  Sasoor 
U.  S.  Aroy  Research  Institute  for  the 
Dehavioral  and  Social  Sciences 
3001  Eisenbouer  Avenue 
Alexandria.  VA  22333 

1  Dr.  Joyce  Shields 
Arty  Research  Institute  for  the 
Bshav'oral  and  Social  Sciences 
5001  Eisenhover  Avenue 
Alexandria.  VA  2233: 

1  Dr.  Hilda  Ring 
Arty  Research  Institute 
SOCl  Eisenhouer  Ave. 

Alexandria,  VA  22333 


Air  Force 

1  Dr.  Earl  A.  Alluisi 
HR,  AFHRL  (AFSC) 
Drooks  AFi,  TI  78235 


1  Nr.  Rayoond  E.  Christal 
RFHRL/NOE 

Drooks  AFD,  Tl  78235 


I  Dr.  Alfred  R.  Fregly 
AFOSR/NL 

Dolling  AFB,  DC  20332 

1  Dr.  Senevieve  Haddad 
^rcgrat  Harager 
;.;(e  Sciences  Drectorate 
AFCSR 

Selling  AFB,  DC  20332 

1  Dr.  Patrick  Ryllonen 
AFHRL /HOE 


1  Nr.  Randolph  Park 
RFNRL/NOAM 

Drooks  AFB,  TI  78235 

1  Dr.  Roger  Pennell 
Air  Force  Huaan  Resources  Laboratory 
LOM-y  AFB,  CO  80230 

1  Dr.  Nalcola  Ree 
AFHRL/HP 

Brooks  AFB,  TI  78235 

1  Hajor  John  Nelsh 
AFHRL/NOAN 

Drooks  AFD  ,  TI  78223 


Departrent  of  Defense 

12  Defense  Technical  Inforoation  Center 
Cateron  Station,  Bldg  5 
Alexandria,  VA  22314 
Attn:  TC 

1  Nilitary  Assistant  for  Training  and 
frsonnel  Technology 
Office  of  the  Under  Secretary  of  Oefens 
for  Research  I  Engineering 
Root  33129,  The  Pentagon 
Mashington,  DC  2C30I 

i  Dr.  H.  Steve  Selloan 
Office  of  the  Assistant  Secretary 
of  Defense  (HPA  I  L) 

2B269  The  Pentagon 
Mashington,  DC  2030! 

1  Dr.  Robert  A.  Nisher 
OISDRE  (ELS 

The  Pentagon,  Rooa  3D129 
Mashington,  DC  20301 


Drooks  AFB,  Tl  78235 
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CivlllM  AgtOClca 

1  Or.  Virn  H.  Urry 
Pirsoflntl  OtO  Ctnttr 
IHfict  o4  Ptrsonntl  Nanagmnt 
IfOO  E  Strtft  m 
tlat^in|ton,  K  20413 

1  Hr.  TkOMi  A.  Mart 
U.  S.  Coait  Suard  Inatituta 
P.  0.  Substation  IB 
Oklahoaa  City,  OK  73169 

1  Dr.  Jostph  L.  Young,  Director 
Heaory  I  Cognitive  Processes 
National  Science  Foundation 
Hashington,  DC  20550 
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Private  Sector 

1  Br.  Jaaes  Algina 

university  of  Florida 

Bainesville,  FL  32605 

1  Br.  Erling  B.  Andersen 

• 

Beparteent  of  Statistics 

Itudiestraede  6 

1433  Cepenhagen 

J 

BENHARK 

.  1  -J 

1  Or.  .iaac  leja'- 

Educational  Testing  Service 

Princeton,  NJ  0fl43C 

_.v3 

1  Dr.  Henucha  Birenbaua 

■  vj 
*• 

School  of  Education 

■ 

Tel  Aviv  University 

• 

Tel  Aviv,  Raeat  Aviv  69976 

Israel 

1  Or.  Nerner  Sirke 

•W 

•  .  ’ 

Personalstaeaaat  der  Bundesuer' 

D-300C  Koelr-  90 

NEST  6ERNANY 

iii 

1  Dr.  R.  Darrell  Ixk 

.  »•  • 

Beparteent  of  Education 

University  of  Chicago 

Chicago,  IL  60637 

•J 
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I  Nr.  Arnold  lobrer 
SKtion  oF  Psychological  Research 
Caserne  Petits  Chateau 
CRE 

1000  Brussels 
Bilgiua 

1  Dr.  Robert  Brennan 
Aeerican  College  Testing  Prograes 
P.  0.  lo«  168 
loea  City.  lA  32243 

1  Dr.  Blenn  Bryan 
6206  Pee  Road 
Bethesda,  HD  20B17 

1  Dr.  Ernest  R.  Cadotte 
307  stokely 

University  of  Tennessee 
Knoxville.  ’N  3’?16 

1  Dr.  John  B.  Ca"’ol! 

409  Elliott  'd. 

Chapel  Hill,  RC 
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1  Ir.  Notmi)  Cliff 
••ft.  of  Ptycholofy 
Itoiv.  of  So.  California 
llRivirfity  Pa.  k 
Ui  UnftlM,  Cil  V0007 

>  Ir.  Mint  Croib*; 
Elucation  Rtatarcb  Ctnttr 
Oni varsity  of  Ltyftn 
loorbaavdaan  2 
2334  EH  Layden 
Tha  NETHEPLAHDS 

1  laa  Cronbact 
IS  Laburnua  Road 
Atharton,  CA  9^205 

I  CTI/Hc6rat-Hill  Library 
2300  Gar dan  Road 
Honta'’ay,  CA  ?j940 

1  Dr.  Haltar  Cunningbai 
Uni  varsity  of  Hiaoi 
••partoant  of  Psychology 
Sainesvilla,  FL  32611 

1  Dr.  Dattpradad  Divgi 
Syracusa  Uni  varsity 
••partoant  of  PsycAoiogr 
Syracusa,  ME  33210 

I  Ir.  Eooanual  lonchin 
•apartaont  of  Psychology 
(Riivorflty  of  Illinois 
OMapaign,  IL  *1820 

1  Ir.  Hai-Xi  long 
•all  Foundation 
Rooa  314,  luilding  I 
•00  Roosevalt  Road 
Gian  Ellyn,  IL  *0137 

1  Or.  Frit:  Drasgoa 
Oapa'tKnt  of  Psychology 
Univarsity  of  Illinois 
6{)3  E.  Danial  St. 
Chaapaign,  IL  61S20 

1  Dr.  SuSi'  Eafcarlscn 
fsrcHGLOS'-  rEP4'‘:PE^' 


1  ERIC  Facility-Acquisitions 
4833  Rugby  Avanua 
•atbasda,  HD  20014 

1  Or.  8anjaaia  A.  FairAank,  ir. 
McFann-lray  t  Asaociatas,  Inc. 

923  Callaghan 
Suita  225 

San  Antonio,  Tl  78228 

1  Dr.  Laorard  Faldt 
Lindquist  Carta'  ir  Baasuraant 
Univarsity  o*  Ion 
loaa  City.  lA  32241 

1  Ur;v.  Prof.  Or.  Sarhard  Fischar 
Liabiggassa  3/3 
A  1010  Vianna 
AUSTRIA 

1  Professor  Honald  Fitigarald 
Univarsity  of  Ha*  England 
Araidala,  Rau  SOkth  Hal as  2351 
AUSTRALIA 

1  Or.  Daitar  Flatcbar 
University  of  Oregon 
Oapartaant  of  Co^utar  Science 
Eugene,  DR  P7403 

1  Or.  John  R.  Fradariksan 
•olt  laranak  *  Rauaan 
SO  Houlton  Street 
Caabridga,  HA  02138 

1  Dr.  Janice  Gifford 
Univarsity  of  Hassachusatts 
School  of  Education 
Aaharst,  HA  01002 

1  Dr.  Robert  Slasar 

Learning  Research  *  Oevalopaent  Center 
University  of  Pittsburgt 
3'»39  O’Hara  Street 
PITTSIURSN,  PA  15260 

1  Dr.  Harvin  D.  Slock 
217  Stone  Hall 
Cornell  University 
Ithaca,  MY  14853 
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1  tr.  Itrt  Srtcn 
Johns  Hopkins  University 
Departoont  of  Psychology 
CbarlH  I  34th  Street 
laitioore,  HS  21218 

1  M.  JMES  6.  6REEM 
LHK 

UHIVERSITY  OF  P1TTSIUR6H 
3439  O’HARA  STREET 
PIT^SBURSH,  PA  15213 

1  Dr.  Ron  Haobleton 
School  of  Education 
University  of  Hassachusetts 
Aeherst,  HA  01002 

1  Or.  Paul  Horst 
i77  6  Street,  1184 
Chwli  Vista.  CA  9001C 

1  Dr.  Llovd  Huipbrevs 
Depa''taent  cf  hsychslog, 

University  of  Illinois 
807  East  Daniel  Street 
Chaepaign,  IL  81020 

1  Dr.  Steven  Hunka 
Departoent  of  Education 
University  of  Alberta 
Edoonton,  Alberta 

CANADA 

1  Ir.  Earl  Hunt 
Dept,  of  Psychology 
University  of  Nashington 
Seattle,  M  9S10S 

1  Dr.  Jack  Hunter 
2122  Coohdge  St. 

Lansing,  HI  48908 

1  Dr,  Huynh  Huynh 
College  of  Education 
University  of  South  Carolina 
Coluebia,  SC  29208 

1  Dr.  Douglas  H.  Jones 
Advanced  Statistical  Technologies 
Corporation 
10  Trafalgar  Cou''t 
Lau'enceville,  HI  08:46 


1  Dr.  Harcel  Just 
Departoent  of  Psychology 
Carnegie-Hellon  University 
Pittsburgh,  PA  1S213 

1  Dr.  Doeetrios  Karis 
Departoent  of  Psycho! ogy 
University  of  Illinois 
803  E.  Daniel  Street 
Chaepaign,  IL  81820 

1  Professor  John  A.  Keats 
Departaent  of  Psychology 
The  University  of  Neucastle 
N.S.M.  2308 
AUSTRALIA 

1  Or.  Milliao  Koch 
University  of  Tevas-Austin 
Neasureoent  and  Evaluation  Center 
Austin,  Tt  78703 

1  Dr.  Alan  Lesgold 
Learning  R80  Center 
University  of  Pittsburgh 
3939  O’Hara  Street 
Pittsburgh,  PA  15280 

1  Or.  Hichael  Levine 
Departoent  of  Educational  Psychology 
210  Education  Dldg. 

University  of  Illinois 
Chaepaign,  IL  81801 

1  Dr.  Charles  Leeis 
FKolteit  Sociale  Hetenschappen 
Rijksuniversiteit  Broningen 
Oude  Boteringestraat  23 
97126C  Broningen 
Netherlands 

1  Dr.  RcRiert  Linn 
College  of  Education 
University  of  Illinois 
Urbana,  IL  81801 

1  Hr.  Phillip  Livingston 
Systees  and  Applied  Sciences  Corpcratic 
861 :  Keniluorth  Avenue 
Riverdale.  HD  2084C 


1  Dr.  Robtrt  Lockun 
Center  tor  Naval  Analysis 
200  North  Beauregard  St. 

Alexandria,  VA  22311 

1  Dr.  Frederic  R.  Lord 
Educational  Testing  Service 
Princeton,  NJ  08341 

1  Dr.  Jaees  Luasder 
Defarteent  of  Psychology 
University  of  Nesterr  Aust'il.a 
Nedlands  H.A.  iOOP 
AUSTRALIA 

1  Or.  Don  Lyor, 

P.  0.  Bo>  4< 

Higley  ,  AZ  8523f> 

1  Dr.  Gary  Harco 
Stop  31-E 

Educational  ^estin;  Ser.ice 
Princeton,  NJ  0843! 

1  Dr,  Scott  Kaxtell 
Departeent  of  Psychology 
University  of  Notre  Oaee 
Notre  Oaae,  IN  WZb 

1  Dr.  Saauel  T.  Nayc 
Loyola  University  of  Chicago 
120  North  Hichigan  Avenue 
Chicago,  IL  Will 

1  Nr.  Robert  HcKlnley 
Noerican  College  Testing  Prograos 
P.O.  Dox  la8 
Iowa  City,  lA  32243 

1  Dr.  Barbara  Heans 
Huoan  Resources  Research  Organization 
300  North  Nashington 
Alexandria,  VA  22314 

1 

Processor  Jason  Nil  lean 
Departeent  c'  Education 
Stone  Hall 
Corbel;  lir.iversit* 

If  ica.  Nv  1403: 


1  Dr.  N.  Alan  Niceeander 
University  of  Otlahoea 
Departeent  of  Psychology 
Oklahoea  City,  OF  730i9 

1  Dr.  Donald  A  Noroan 
Cognitive  Science,  C-013 
Univ.  of  California,  San  Diego 
La  Jolla,  CA  92093 

1  Dr.  Ne'.vin  R.  Novic> 

336  Lindquist  Cente'  for  Heasureent 
Uni.ersity  of  loea 
loea  City,  lA  32242 

1  Dr.  Janes  Olson 
NICAT,  Inc. 

1873  Scuth  State  Street 
Orea,  UT  84037 

1  Hayne  N.  Patience 
Anerican  Cccnci'.  on  Education  . 
BED  Testing  Service,  Suite  20 
One  Dupont  Cirle,  NN 
Nashington,  DC  20036 

1  Dr.  Janes  Paulson 
Dept,  of  Psychology 
Portland  State  UniveSrsity 
P.O.  Box  731 
Portland,  OR  97207 

1  Or.  Janes  H.  Pellogrino 
University  of  California, 

Santa  Barbara 
Dept.  of  Psychology 
Santa  Darabara  ,  CA  93106 

1  Dr.  Douglas  H.  Jones 
Advanced  Statistical  Technologies 
Corporation 
10  Trafalgar  Court 
Laerenceville,  NJ  08148 

1  Dr.  Steven  E.  FoltrocL 
Bell  Laboratories  2D-444 
60C  Nountain  Ave. 

Hur'-av  Hii;,  NJ  07974 

1  Dr.  Na-'k  D.  Reciase 
ACT 

P.  0.  Bor  168 
love  City,  lA  32243 


1  D'.  Robert  Hislew 
711  lilincis  Street 
Geneva.  IL  6C!!4 


I  Ir.  Thoui  Rtynolds 
Univtriity  of  Tnat-Oillti 
lUrketing  ItpirtMnt 
P.  0.  loi  686 
Rickariton,  TX  79080 

1  8r.  LawMCt  Radntr 
403  Elt  Ovinut 
T*koM  Park,  m  20012 

1  Dr.  J,  Ryan 
Dtpartaent  of  Education 
University  of  South  Carolina 
Coluabia,  SC  29208 

1  PROF.  FUHIKO  SANEJIHA 
DEPT.  OF  PSYCHOLOBY 
UNIVERSITY  OF  TENNESSEE 
KNOIVILLE,  TN  3^916 

1  Frank  L.  Schaidt 
Departient  of  Psychology 
Rldg.  66 

George  Mashington  University 
Nashington,  DC  200C2 

1  Or.  Halter  Schneider 
Psychology  Departaent 
603  E.  Daniel 
Chaapaign,  IL  61820 

1  Leaell  Schoer 
Psychological  6  Quantitative 
Foundations 
College  of  Education 
University  of  loaa 
loaa  City,  lA  S2242 

I  Dr.  Eaaanuel  Donchin 
Departaent  of  Psychology 
University  of  Illincis 
Chaapaign,  IL  61820 

I  Dr.  Kaiuo  Shigenasu 
7*9' 24  Kugenuaa-F.aigan 
Fujusana  251 
JAPAN 

1  Cr.  Hilliaa  Sias 
Cente*"  for  Na«a!  final  vs;  s 
200  North  Beaurec.rd  Street 
filekandna,  VA  21!!! 


1  Dr.  H.  Nallace  Sinaiko 
Prograa  Director 

Nanpoaer  Research  and  Advisory  Services 
Saithsonian  Institution 
801  North  Pitt  Street 
Alexandria,  VA  22314 

1  Martha  Stocking 
Educational  Testing  Service 
Princeton,  NJ  08541 

1  Dr.  Peter  Stolof* 

Center  for  Naval  Analysis 
200  North  Beauregard  Street 
Alexandria,  VA  22311 

1  Dr.  Hilliaa  Stout 
University  of  Illinois 
Departaent  of  Natheaatics 
Urbana,  IL  61801 

1  DR.  PATRICK  SUPPES 
INSTITUTE  FOR  HATHEHATICA.  STUOIES  IN 
THE  SOCIAL  SCIENCES 
STANFORD  UNIVERSITY 
STANFORD,  CA  94305 

1  Dr.  Hariharan  Saaainathan 
Laboratory  of  Psychoaetric  and 
Evaluation  RHearch 
School  of  Education 
UnivKsity  of  Massachusetts 
Aaherst,  NA  01003 

1  Dr.  Kikuai  Tatsuoka 
Coaputar  Based  Education  Research  Lab 
252  Engineering  Research  Laboratory 
Urbana,  IL  61801 

1  Or.  Maurice  Tatsuoka 
220  Education  Bldg 
1310  S.  Sixth  St. 

Chaapaign,  IL  61820 

I  Dr.  David  Thissen 
Departient  of  Psychology 
University  of  Kansas 
Laurence,  KS  66044 

1  Dr.  Douglas  Toune 
Univ.  of  Sc.  Cali^cnia 
Behavioral  TethnoJc^,  ljijs 
1845  S.  Elena  Avt. 

Redondc  Bea:“.  Cfi  91277 


c 


.’v/^  1  #r.  Robtrt  T«ttt*ki>«i 
v  oi  Stitiitic* 

0»iv#r»ity  of  Hitiouri 
•  ColuabK,  HO  45201 

^  1  If.  LHyifd  Tocktr 
UBivoriity  of  Illinoi* 
Otpirtoont  of  Piycholooy 
603  E.  0«i*l  Strut 
Chaopiign,  IL  61B2C 


1  Or.  V.  R.  R.  Uppuluri 
Union  Carbide  Corporation 
Hue  1  ear  Diviiion 
P.  0.  lo>  Y 
Oak  Ridge,  TN  37830 

1  Or.  David  Vale 
Assesownt  Systees  Corporation 
2233  University  Avenue 
Suite  310 
St.  Paul,  HH  SSIM 

1  Dr.  Houard  Maine' 

Division  of  Psychological  Studies 
Educational  Testing  Service 
Princeton,  M  08540 

1  Or.  HicAael  T.  Haller 
Departoent  of  Educational  Psychology 
University  of  Hisconsin— Hiluaukee 
UilMukK,  Ml  53201 

1  Dr.  Irian  Haters 
HuaRM 

300  Morth  Hashington 
Aletandria,  VA  ^314 

1  Or.  David  J.  Heiss 
H660  Elliott  Mall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  HN  5S4SS 


1  Or.  Rand  R.  ll;lco« 

University  of  Southern  Califc'nia 
Decartnent  of  Psychology 
Lcs  Angeles,  CA  P0CC7 


1  lerean  Military  Representative 
RTTH:  Hoi f gang  Hildegrube 
Strcitkraefteaet 
I-S300  knn  2 

4400  Irandyuine  Itreet,  MH 
Uasbington  .  K  20016 

1  Dr.  Iruce  HilliaH 
Departeent  of  Educational  Psychology 
University  of  Illinois 
Urbana,  IL  61801 

1  Ms.  Marilyn  Hingersky 
Educational  Testing  Service 
Princeton,  NJ  0854; 

1  Or.  Hendy  Yen 
CTI/Mc6ra«  Mill 
Del  Monte  Research  Park 
Monterey,  CA  93940 
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